With Smart Cameras

Prerequisites

This guide assumes you already know the Alchemist metamodel and how to write simulations in YAML.

Smartcam

A smartcam is a camera able to detect objects of interest and to communicate with other smartcameras. In many cases it is also assumed to be mounted on a drone as to be able to freely move around in the environment. In Alchemist smartcams are simulated as Nodes equipped with specific Reactions defining their capabilities and behaviour. Single capabilities are expressed as Actions.

Vision

The most basic example of a camera is a Node containing a Reaction with the CameraSee action. Note: the CameraSee action currently only works in 2D environments supporting euclidean geometry, for example Continuous2DEnvironment and Rectangular2DEnvironment. The CameraSee action requires 3 parameters to be defined in this order: the distance of the field of view, its angle in degrees, and the name of the Molecule which will contain the ouput, namely a list of the nodes contained in the field of view which is updated each time the action is triggered. Optionally a fourth parameter can be defined in order to filter the output. Such parameter is expected to be the name of a Molecule which has to be contained in a Node for it to be visible, e.g. if it is "wanted" then only nodes containing a molecule named "wanted" will be seen.

Movement

The ability to move can be defined using movement actions such as MoveToTarget or FollowAtDistance. MoveToTarget expects 2 parameters such as the name of the molecule containing the target's position and the movement speed. FollowAtDistance requires the name of the molecule containing the target's position, the distance to mantain from the target, and the movement speed.

Rotation

The action HeadTowardTarget can be used to instruct cameras to always face the specified target, it requires 2 parameters such as the name of the molecule containing the target's position and the angular speed in degrees. The Spin action only requires the angular speed and will make the camera spin around itself like a radar.

Algorithm

Without defining an algorithm the cameras wouldn't do anything interesting. Algorithms can be definied in a moltitude of different ways. Below there's an example of a basic algorithm defined only with Reactions, Conditions and Actions. It is advisable to use real programming languages such as Protelis to write more elaborate algorithms. In this regard you'd want to use the protelis incarnation, make cameras be ProtelisNode, and to make sure that the input and output molecules' names of the actions match the ones used by the protelis program.

Complete example of a simulation

We start by writing a basic yaml configuration to place 20 potential targets and 10 cameras in a 400 x 400 rectangular environment:

incarnation: protelis

variables:
  NumHumans: &NumHumans
    formula: 20
  NumCameras: &NumCameras
    formula: 10
  EnvironmentSize: &EnvironmentSize
    formula: 400
  HalfEnvironmentSize: &HalfEnvironmentSize
    formula: EnvironmentSize / 2
  EnvironmentOrigin: &EnvironmentOrigin
    formula: -HalfEnvironmentSize


environment:
  type: Rectangular2DEnvironment
  parameters: [*EnvironmentSize, *EnvironmentSize]

deployments:
  - in: # possible targets
      type: Rectangle
      parameters: [*NumHumans, *EnvironmentOrigin, *EnvironmentOrigin, *EnvironmentSize, *EnvironmentSize]
    nodes:
      type: CircleNode
      parameters: [1]
  - in: # cameras
      type: Rectangle
      parameters: [*NumCameras, *EnvironmentOrigin, *EnvironmentOrigin, *EnvironmentSize, *EnvironmentSize]
    nodes:
      type: ProtelisNode

Then we give cameras the ability to see other nodes. The CameraSee action requires the distance and angle of the field of view. Each time it is triggered, it writes all the nodes seen in the molecule vision.

incarnation: protelis

variables:
  NumHumans: &NumHumans
    formula: 20
  NumCameras: &NumCameras
    formula: 10
  EnvironmentSize: &EnvironmentSize
    formula: 400
  HalfEnvironmentSize: &HalfEnvironmentSize
    formula: EnvironmentSize / 2
  EnvironmentOrigin: &EnvironmentOrigin
    formula: -HalfEnvironmentSize
  CameraFoVAngle: &CameraFoVAngle
    formula: 60
  CameraFoVDistance: &CameraFoVDistance
    formula: 20

environment:
  type: Rectangular2DEnvironment
  parameters: [*EnvironmentSize, *EnvironmentSize]

programs:
  - &Camera
    - time-distribution: 1
      type: ChemicalReaction
      actions:
        - type: CameraSee
          parameters: [*CameraFoVDistance, *CameraFoVAngle, vision]

deployments:
  - in: # possible targets
      type: Rectangle
      parameters: [*NumHumans, *EnvironmentOrigin, *EnvironmentOrigin, *EnvironmentSize, *EnvironmentSize]
    nodes:
      type: CircleNode
      parameters: [1]
  - in: # cameras
      type: Rectangle
      parameters: [*NumCameras, *EnvironmentOrigin, *EnvironmentOrigin, *EnvironmentSize, *EnvironmentSize]
    nodes:
      type: ProtelisNode
    programs:
      - *Camera

In this example we make humans move randomly. We also want to randomly toggle a wanted molecule in them, as to have a way to mark interesting targets that needs to be followed by the cameras. Every 50 seconds each human has a 1% probability to become wanted.

incarnation: protelis

variables:
  HumanSpeed: &HumanSpeed
    formula: 1
  NumHumans: &NumHumans
    formula: 20
  NumCameras: &NumCameras
    formula: 10
  EnvironmentSize: &EnvironmentSize
    formula: 400
  HalfEnvironmentSize: &HalfEnvironmentSize
    formula: EnvironmentSize / 2
  EnvironmentOrigin: &EnvironmentOrigin
    formula: -HalfEnvironmentSize
  CameraFoVAngle: &CameraFoVAngle
    formula: 60
  CameraFoVDistance: &CameraFoVDistance
    formula: 20

environment:
  type: Rectangular2DEnvironment
  parameters: [*EnvironmentSize, *EnvironmentSize]

programs:
  - &Human
    - time-distribution: 1
      type: ChemicalReaction
      actions:
        - type: ConstantDistanceRandomMove
          parameters: [*HalfEnvironmentSize, *HumanSpeed]
    - time-distribution: 0.02
      type: ChemicalReaction
      actions:
        - type: RandomlyToggleMolecule
          parameters: [wanted, true, 0.01]
  - &Camera
    - time-distribution: 1
      type: ChemicalReaction
      actions:
        - type: CameraSee
          parameters: [*CameraFoVDistance, *CameraFoVAngle, vision]

deployments:
  - in: # possible targets
      type: Rectangle
      parameters: [*NumHumans, *EnvironmentOrigin, *EnvironmentOrigin, *EnvironmentSize, *EnvironmentSize]
    nodes:
      type: CircleNode
      parameters: [1]
    programs:
      - *Human
  - in: # cameras
      type: Rectangle
      parameters: [*NumCameras, *EnvironmentOrigin, *EnvironmentOrigin, *EnvironmentSize, *EnvironmentSize]
    nodes:
      type: ProtelisNode
    programs:
      - *Camera

It is time for cameras to identify wanted targets so we add a CameraInjectVisibleNodeClosestToDistance action in order to select the closest node to the center of the field of view. We also specify the wanted molecule in the CameraSee action in order to filter only the interesting nodes. Then, at the condition that a target is found, we want the cameras to start following it, so we add the actions FollowAtDistance and HeadTowardTarget.

incarnation: protelis

variables:
  HumanSpeed: &HumanSpeed
    formula: 1
  NumHumans: &NumHumans
    formula: 20
  NumCameras: &NumCameras
    formula: 10
  EnvironmentSize: &EnvironmentSize
    formula: 400
  HalfEnvironmentSize: &HalfEnvironmentSize
    formula: EnvironmentSize / 2
  EnvironmentOrigin: &EnvironmentOrigin
    formula: -HalfEnvironmentSize
  CameraFoVAngle: &CameraFoVAngle
    formula: 60
  CameraFoVDistance: &CameraFoVDistance
    formula: 20
  CameraSpeed: &CameraSpeed
    formula: 1
  CameraAngularSpeed: &CameraAngularSpeed
    formula: 3
  CameraDistanceFromTarget: &CameraDistanceFromTarget
    formula: 2 * CameraFoVDistance / 3

environment:
  type: Rectangular2DEnvironment
  parameters: [*EnvironmentSize, *EnvironmentSize]

programs:
  - &Human
    - time-distribution: 1
      type: ChemicalReaction
      actions:
        - type: ConstantDistanceRandomMove
          parameters: [*HalfEnvironmentSize, *HumanSpeed]
    - time-distribution: 0.02
      type: ChemicalReaction
      actions:
        - type: RandomlyToggleMolecule
          parameters: [wanted, true, 0.01]
  - &Camera
    - time-distribution: 1
      type: ChemicalReaction
      actions:
        - type: CameraSee
          parameters: [*CameraFoVDistance, *CameraFoVAngle, vision, wanted]
        - type: CameraInjectVisibleNodeClosestToDistance
          parameters: [*CameraDistanceFromTarget, vision, target]
    - time-distribution: 1
      type: ChemicalReaction
      conditions:
        - type: ContainsMolecule
          parameters: [target]
      actions:
        - type: HeadTowardTarget
          parameters: [target, *CameraAngularSpeed]
        - type: FollowAtDistance
          parameters: [target, *CameraDistanceFromTarget, *CameraSpeed]

deployments:
  - in: # possible targets
      type: Rectangle
      parameters: [*NumHumans, *EnvironmentOrigin, *EnvironmentOrigin, *EnvironmentSize, *EnvironmentSize]
    nodes:
      type: CircleNode
      parameters: [1]
    programs:
      - *Human
  - in: # cameras
      type: Rectangle
      parameters: [*NumCameras, *EnvironmentOrigin, *EnvironmentOrigin, *EnvironmentSize, *EnvironmentSize]
    nodes:
      type: ProtelisNode
    programs:
      - *Camera

To conclude, we want the cameras to explore randomly and spin when no targets are detected. For this purpose we add another reaction with a Spin and ConstantDistanceRandomMove and with an NoOtherReactionCanExecute condition which will be triggered when the conditions of the other reactions are not valid.

incarnation: protelis

variables:
  HumanSpeed: &HumanSpeed
    formula: 1
  NumHumans: &NumHumans
    formula: 20
  NumCameras: &NumCameras
    formula: 10
  EnvironmentSize: &EnvironmentSize
    formula: 400
  HalfEnvironmentSize: &HalfEnvironmentSize
    formula: EnvironmentSize / 2
  EnvironmentOrigin: &EnvironmentOrigin
    formula: -HalfEnvironmentSize
  CameraFoVAngle: &CameraFoVAngle
    formula: 60
  CameraFoVDistance: &CameraFoVDistance
    formula: 20
  CameraSpeed: &CameraSpeed
    formula: 1
  CameraAngularSpeed: &CameraAngularSpeed
    formula: 3
  CameraDistanceFromTarget: &CameraDistanceFromTarget
    formula: 2 * CameraFoVDistance / 3

environment:
  type: Rectangular2DEnvironment
  parameters: [*EnvironmentSize, *EnvironmentSize]

programs:
  - &Human
    - time-distribution: 1
      type: ChemicalReaction
      actions:
        - type: ConstantDistanceRandomMove
          parameters: [*HalfEnvironmentSize, *HumanSpeed]
    - time-distribution: 0.02
      type: ChemicalReaction
      actions:
        - type: RandomlyToggleMolecule
          parameters: [wanted, true, 0.01]
  - &Camera
    - time-distribution: 1
      type: ChemicalReaction
      actions:
        - type: CameraSee
          parameters: [*CameraFoVDistance, *CameraFoVAngle, vision, wanted]
        - type: CameraInjectVisibleNodeClosestToDistance
          parameters: [*CameraDistanceFromTarget, vision, target]
    - time-distribution: 1
      type: ChemicalReaction
      conditions:
        - type: ContainsMolecule
          parameters: [target]
      actions:
        - type: HeadTowardTarget
          parameters: [target, *CameraAngularSpeed]
        - type: FollowAtDistance
          parameters: [target, *CameraDistanceFromTarget, *CameraSpeed]
    - time-distribution: 1
      type: ChemicalReaction
      conditions:
        - type: NoOtherReactionCanExecute
          parameters:
      actions:
        - type: Spin
          parameters: [*CameraAngularSpeed]
        - type: ConstantDistanceRandomMove
          parameters: [*HalfEnvironmentSize, *CameraSpeed]

deployments:
  - in: # possible targets
      type: Rectangle
      parameters: [*NumHumans, *EnvironmentOrigin, *EnvironmentOrigin, *EnvironmentSize, *EnvironmentSize]
    nodes:
      type: CircleNode
      parameters: [1]
    programs:
      - *Human
  - in: # cameras
      type: Rectangle
      parameters: [*NumCameras, *EnvironmentOrigin, *EnvironmentOrigin, *EnvironmentSize, *EnvironmentSize]
    nodes:
      type: ProtelisNode
    programs:
      - *Camera

Further references

Lukas Esterle, Peter R. Lewis
Online Multi-object k-coverage with Mobile Smart Cameras
In Proceedings of the International Conference on Distributed Smart Cameras (ICDSC). Nominated for best paper. 2017.