The smartphone sensing blocks are the most numerous in Robobo. All of them are related to information that Robobo can capture through the sensors located in the smartphone, and therefore, are those that provide the robot with a greater potential. It is important to use the monitoring window to check the values returned by these sensors in real time in each specific environment.

We can organize these blocks into 5 groups, depending on the sensor on which they are based:

• CAMERA:

• MICROPHONE:

• SCREEN:

• GIROSCOPE AND COMPASS:

• BATTERY:

Tap Zone

• FUNCTION: Returns the area of the smartphone screen on which a brief press has been made (tap). This block only detects simple pulsations, that is, if we briefly press the screen twice, it will return two values.
  • RANGE:  Returns one of the following five values: forehead, eye, right, left, mouth or chin, depending on the area of the screen on which the pulsation is made. The following diagram shows, approximately, the zones corresponding to the values returned.
  • WARNING: This sensor maintains the last measured value.

• FIELDS:
  • No fields.
• EXAMPLE:

The following program makes Robobo show “anger” if you put your finger in its eye. To do this, it checks if the tap zone block returns “eye”, in which case the facial expression is changed to angry, the wheels are moved from side to side and, finally, the expression returns to normal.

Tap position at _ axis

• FUNCTION: Returns the position x or y in which a brief screen pulsation (tap) is detected.
  • WARNING: This sensor maintains the last measured value.
  • RANGE: From 0 to 100 in both axes, as shown in the following image:

• FIELDS:
  • AXIS SELECTION: 
    • Range: There are 2 possible axes to be selected, x and y, which correspond to the Cartesian coordinates shown above.

• EXAMPLE:

The following program makes Robobo say “I have no mustache” by briefly pressing on the area of the screen between the mouth and the eye. It is necessary to reset the value of the tap sensor in each iteration of the loop because this sensor maintains the last measured value:

Fling Angle

• FUNCTION: Returns the angle at which you slide your finger across the screen of the smartphone (fling). This angle is given by the direction of the gesture on the screen.
  • WARNING: This sensor maintains the last measured value.
  • RANK: From 0 to 360 degrees, in polar coordinates. The following images show 3 possible fling gestures that could be made on the smartphone screen, and the angle returned by this sensor. In the left image, the angle would be the same in both gestures, 180 degrees. In the central image, the angle would be about 70 degrees and, finally, in the right image the angle would be about 330 degrees.

• FIELDS:
  • No field.
• EXAMPLE:

Below is a program that moves the PAN and TILT motors of Robobo according to the fling angle that is made on the screen. For example, if a 30-degree fling is performed, Robobo will move the PAN angle to the left (move pan to 100 at speed 5):

Brightness

• FUNCTION: Returns the ambient light level detected by the built-in sensor on the smartphone screen.
  • WARNING: Not all smartphone models include this sensor.
  • RANGE: Any positive integer value. The value is returned in LUX (lx), the international system unit for the lighting level.

• FIELDS:
  • No field.

• EXAMPLE OF OPERATION:

The following program causes Robobo to turn on itself while the level of illumination on the screen exceeds 1000 lx. Otherwise, it stops:

Last note

• FUNCTION: Returns the last musical note detected with the microphone of the smartphone. The note is continuously measured.
  • RANGE: Returns the musical note in text format, according to the English nomenclature, that is:

• FIELDS:
  • No field.
• EXAMPLE:

The following program consists of two parallel execution threads, that is, when pressing on the icon of the flag both will be executed at the same time. The one on the left generates musical notes from MI to RE following the numeric notation shown in the block play note _ for _ seconds. The one on the right, detects emitted notes with the block last note and turns on the LEDs of the Robobo base based according to the detected note. When executing this program, it can be observed that for some notes (sharped), no LED lights up:

when note detected

• FUNCTION: It is activated when a note is detected. Its operation is that of any Scratch hat block, that is, if this block is activated, those that are connected to it are executed.
  • WARNING: This block is continuously running, capturing all the sounds detected by the smartphone. Its use can slow down other blocks that use sound.

• FIELDS:
  • No fields.
• EXAMPLE:

The following program makes that, when an DO is detected, Robobo informs the user by voice. This type of block is usually used with others in parallel, so that the robot is doing different things depending on the sensor:

Clap Counter

• FUNCTION: Returns the number of claps detected by the smartphone’s microphone.
  • RANGE: Any positive integer number, since it counts the claps that it detects in a cumulative way
    
  • WARNING: The block is set for the detection of clap-type sounds, but it must be taken into account that the ambient noise greatly affects the detection, being able to confuse similar noises in this sensor. On the other hand, this sensor maintains the last detected value.
• FIELDS:
  • No fields.
• EXAMPLE:

The following program detects when two or more consecutive claps are made (in less than two seconds), which makes Robobo emit a sound of joy and put the surprise expression. To do this, a timer is used, which restarts every 2 seconds the clap counter block. If we emit a single slap, nothing happens. It is a simple example of the type of interaction that can be achieved with Robobo when using sound:

Face Distance

• FUNCTION: Returns the distance at which the smartphone front camera detects a face. 
  • RANGE: returns only three text values: close, mid (medium) or far.
  • WARNING: the face detection depends a lot on the ambient light level. It is recommended that the face be properly illuminated. On the other hand, the precision in the face detection depends on the quality of the smartphone front camera. 
• FIELDS:
  • No fields
• EXAMPLE OF OPERATION:

The following example makes Robobo look for a nearby face by moving the PAN motor. When it finds the face, it stops and says “I see you”. To do this, a counter variable is used to identify the state (1, 2 or 3) in which the PAN is located, and to turn it accordingly:

Face position at _ axis

• FUNCTION: Returns the x or y position in which a face is detected by the smartphone front camera. The returned point (x, y) corresponds to the midpoint between the two eyes of the detected face:
  • WARNING: The face detection depends a lot on the ambient light level and how it affects the face to be detected. It is recommended that the face be properly illuminated. On the other hand, the precision in the face detection depends on the quality of the smartphone front camera. This sensor stores the last detected value. 
  • RANGE: From 0 to 100 in both axes, using the camera perspective, as shown in the following image, where the coordinates of the detected face would be (40, 60):

• FIELDS:
  • AXIS SELECTION: 
    • Range: there are 2 possible axes to be selected, x and y, which correspond to the Cartesian coordinates shown above.
• EXAMPLE:

The following program makes Robobo say where it detects a face in a natural way for the user. So, when it detects a face to the right, left or centered, it says so. If it does not detect any face, it says “I can’t see you”. To do this, it uses the x coordinate of the detected face:

when FACE IS detected

• FUNCTION: It is activated when a face is detected. Its operation is that of any Scratch hat block, that is, if this block is activated, those that are connected to it are executed.
  • WARNING: This block is continuously running, continuously capturing the presence of a face. Its use can slow down other blocks that use face detection.

• FIELDS:
  • No fields.

• EXAMPLE:

The following program makes Robobo say “I see you” continuously every 2 seconds whenever its detects a face:

when FACE IS LOST

• FUNCTION: It is activated when a face is no longer detected. Its operation is that of any Scratch hat block, that is, if this block is activated, those that are connected to it are executed.
  • WARNING: This block is continuously running, continuously capturing the non-presence of a face. Its use can slow down other blocks that use face detection.

• FIELDS:
  • No fields.

• EXAMPLE:

The following program makes Robobo say “I see you” and “I can’t see you” continuously every 2 seconds whenever it detects a face and then loses it:

_ BLOB Area

• FUNCTION: Returns the area, of the color selected in the drop-down field, that is detected in the image of the smartphone front camera.
  • WARNING: The detection of the area of a certain color depends very much on the ambient light level and how it affects the object to be detected. It is recommended that the object be properly illuminated. On the other hand, the accuracy in color detection depends on the quality of the smartphone front camera.
  • CALIBRATED: For optimal operation of this block, it is recommended to perform the camera calibration process described in app user guide. During calibration you can also select the custom color.
  • RANK: The returned area can have any positive integer value and it is measured in pixels.
• FIELDS:
  • COLOR SELECTION: The area returned by this sensor will be that of the color selected in this field. If there is more than one zone of the same color in the image, the area of the largest one will be returned, but not the sum of all of them.
    • Range: The possible colors to detect are red, green, blue and custom. By default, the only color detected is green. To detect other colors, they must be previously activated in the block configure blob detection colors.
• EXAMPLE: 

The following program associates the detected area of a certain color object (green in this case) with the distance to that object. Thus, if the area is less than 2000, Robobo accelerates to approach. If it is between 2000 and 8000, it moves more slowly and, finally, if it is greater than 8000 it stops. To test example, it is recommended to use a green object that is placed in front of the robot:

_ Blob position at _ axis

• FUNCTION: Returns the x or y coordinate of the center of the color area selected in the drop-down field detected by the front smartphone camera.
  • WARNING: The detection of the area of a certain color depends very much on the ambient light level and how it affects the object to be detected. It is recommended that the object be properly illuminated. On the other hand, the accuracy in color detection depends on the quality of the front camera of the smartphone.
  • CALIBRATED: For optimal operation of this block, it is recommended to perform the calibration process of the camera described in app user guide. During calibration you can also select what is the custom color.
  • RANK: The output values go from 0 to 100 for both x and y, as shown in the following figure (the example value shown would be the coordinates of the center of the blue area, of values (30,30)):

• FIELDS:
  • COLOR SELECTION: 
    • The value x or y returned by this sensor corresponds to the center of the area of the color area selected in this field. If there is more than one area of the same color in the image, the x or y of the largest one will be returned.
    • Range: The possible colors to detect are red, green, blue and custom. By default, the only color detected is green. To detect other colors, they must be previously activated in the configure blob detection colors block.
  • AXIS SELECTION: 
    • Range: The possible values are x or y, according to the Cartesian coordinates shown in the previous image.
• EXAMPLE OF OPERATION:

The following program makes Robobo say where he detects a green object, in a natural way for the user. So, when it is detected on the right, left or middle, it says so. If it does not detect any green object, it says I can’t see green. For this, the x coordinate of the largest green area detected in the image is used:

Active Blob Detection Colors R _ G _ B _ C _

• FUNCTION: It allows you to select which colors will be detected in the read _ area and read _ blob position at _ axis blocks, using the red, green, blue or custom color activation selection fields. The color detection consumes a lot of smartphone resources, because it is analyzing the camera image continuously, which requires an important use of the processor that can slow down Robobo operation.
  • WARNING: This block affects the program in which it is used, and only while it is running. By default, the only color detected is green.
  • CALIBRATED: For optimal operation of the blocks that use color detection with the camera, it is recommended to perform the calibration process from the smartphone app, described in the app user guide. During calibration you can also select the custom color.
• FIELDS:
  • RED ACTIVATION:
    • Range: There are two possible values, true or false, which indicate respectively if the program in which this block is used should detect the red color or not.
  • GREEN ACTIVATION:
    • Range: There are two possible values, true or false, which indicate respectively if the program in which this block is used should detect the green color or not.
  • BLUE ACTIVATION:
    • Range: There are two possible values, true or false, which indicate respectively if the program in which this block is used should detect the blue color or not.
  • CUSTOM COLOR ACTIVATION:
    • Range: There are two possible values, true or false, which indicate respectively if the program in which this block is used should detect the custom color or not.
• EXAMPLE OF OPERATION:

The following program configures the red and blue colors for detection. Next, by placing an object of each color in front of the camera, Robobo says if it detects red or blue, and turns on the LEDs of the base in each case. By default, the LEDs on the base are green:

Orientation at _ axis

• FUNCTION:  Returns the smartphone orientation according to the navigation angle chosen in the drop-down selection field. As the smartphone is fixed to the base through the holder, this orientation is that of Robobo itself.
  • WARNING: Not all smartphones have an orientation sensor (gyroscope).
  • RANK: From -180 to 180 degrees on all axes, according to the navigation angle scheme of the figure below. It is important to note that the yaw angle turn is achieved by turning the base with the wheel motors and the PAN motor. The turn in the roll angle cannot be achieved by Robobo, but if the surface on which it moves has a lateral inclination, it will change. Finally, you can rotate the pitch angle when using the TILT motor or move the robot through a frontally inclined plane:

• FIELDS:
  • ANGLE SELECTION:
    • Range: There are 3 possible navigation angles: yaw, pitch and roll.
• EXAMPLE:

The following program makes Robobo to move in a straight line indefinitely, and say if it goes up or if it goes down in case of finding an incline in its path. For this, it is analyzed if the pitch angle is above a negative or positive threshold:

Reset sensor _

• FUNCTION: Restarts the sensor value indicated in the sensor selection field. This block also affects the  Robobo base sensors, so its description also appears in that section.
• FIELDS:
  • SENSOR SELECTION: 
    • all: Resets all sensors.
    • IR: Resets the value of the infrared sensors to 0.
    • acceleration: Restarts the value of the accelerometer to 0 on the three axes (x, y, z).
    • brightness: Resets the value of the ambient light sensor to 0.
    • claps: Resets the clap counter to 0.
    • face: Resets the face distance sensor to 0 and the face position sensor (x, y) to (0,0).
    • fling: Restarts the fling angle sensor on the screen to 0.
    • orientation: Restarts the smartphone’s orientation angles (yaw, pitch, roll) to 0.
    • tap: Restarts the tap sensor on the screen to NONE.
• EXAMPLE:

This block is useful for those sensors that store the last value detected or that store a value continuously, for example, the clap sensor or the face distance sensor. The following example shows a program that detects the claps and, while they are less than 5, it turns on the LEDs of the base in blue. As soon as they are 5 or more,  it turns on the LEDs in red and waits for 3 seconds. For the program to continue indefinitely, it is necessary to reset the value of the clap sensor:

Smartphone battery level

• FUNCTION: Returns the smartphone battery level.
  • Range: From 0 to 100, so that the higher the value, the higher the battery level.
• FIELDS:
  • No fields.
• EXAMPLE:

Below is a typical program that could be in permanent execution controlling the battery level of the smartphone every 5 minutes (300 seconds), and that emits a warning when such level falls below a certain threshold. In this case, the warning consists on saying the phrase “Very low battery, save your program” every 30 seconds:

acceleration at _ axis

• FUNCTION: Returns the acceleration suffered by the smartphone on the axis chosen in the axis selection drop-down field. As the smartphone is fixed to the base through the holder, this acceleration is that of Robobo itself.
  • RANGE: Any integer value, positive or negative, measured in m/s². The axes that are used as a reference are the following:

• FIELDS:
  • AXIS SELECTION:
    • Range: You can select the 3 axes x, y, z of the Cartesian system.