pylips.speech.RobotFace
- class pylips.speech.RobotFace(robot_name='default', server_ip='http://localhost:8000', tts_method='system', voice_id=None)[source]
The controllable interface for speech, gaze, and expressions.
- Parameters
- __init__(robot_name='default', server_ip='http://localhost:8000', tts_method='system', voice_id=None)[source]
Methods
__init__([robot_name, server_ip, ...])express(aus, time)Activates a set of Action Units (AUs) on the face.
look(x, y, z, time)Looks to a location in the face's reference frame.
Cancels the current gaze command.
save_file(content, tag)Saves a speech file to the disk.
say(content[, wait, move_face])Issues a speech command to the face.
say_file(tag[, move_face])Plays a speech file that was saved to the disk.
set_appearance(config)Changes the design of the face.
set_component_offsets(offsets[, time])Sets direct displacement offsets for face component control points.
Cancels the current speech command.
stream_file_to_browser(tag[, move_face])Streams an audio file directly to the browser for playback.
wait()Waits for the current speech command to finish.
- express(aus, time)[source]
Activates a set of Action Units (AUs) on the face.
The robot’s face is controlled by a set of Action Units (AUs) that can be activated to create different facial expressions. This method will take in a dictionary of AUs that maps the AU number to the intensity that the AU should be activated at. The method will also take in a time in milliseconds to reach the requested AU state.
Action units can also be unilateral, meaning that they can be activated on one side of the face. For example, AU1 is the inner brow raiser, and activating it on one side of the face can be done by specifying ‘AU01l’ or ‘AU01r’ in the dictionary.
- look(x, y, z, time)[source]
Looks to a location in the face’s reference frame.
The robots references frame has its origin at the center of the eyes, with the positive x-axis pointing to the right, the positive y-axis pointing up, and the positive z-axis pointing out of the face.
TODO: add a check to make sure the location is within the robot’s range of motion
- release_gaze()[source]
Cancels the current gaze command.
If the robot is looking at a location, the robot will stop looking at that location and return to its default gaze behaviors, looking idly in different directions. If the robot is not looking at a location, this method will have no effect.
- Parameters
None –
- save_file(content, tag)[source]
Saves a speech file to the disk.
This method will take in a string of text and create the pkl and sound files necessary to call
say_filelater.
- say(content, wait=False, move_face=True)[source]
Issues a speech command to the face.
This method will take in a string of text and convert it to speech. This method may also shift
self.channelto avoid competing with other connected faces.
- say_file(tag, move_face=True)[source]
Plays a speech file that was saved to the disk.
- Parameters
tag (str) – the tag that was used to save the file. This should be a string with no extensions or folder paths – e.g., ‘statement1’ or ‘question2’
- set_appearance(config)[source]
Changes the design of the face.
Different parameters can be passed to the face to change its appearance. This method will take in a dictionary of configuration parameters that will be used to change the face’s appearance, and update the face to reflect the new configuration.
- Parameters
config (dict) – a dictionary of configuration parameters that will be used to change the face’s appearance – e.g., {‘iris_color’: ‘#00FF00’, ‘eye_size’: 200, ‘eye_height’: 60}
- set_component_offsets(offsets, time=0)[source]
Sets direct displacement offsets for face component control points.
This method allows you to directly control the position of eyebrow and mouth control points by providing displacement vectors. The offsets are applied on top of the default positions and any Action Unit-based adjustments.
- Parameters
offsets (dict) –
A dictionary containing component offsets. Valid keys:
’eyebrow_left’ (list): 6-element list [x1, y1, x2, y2, x3, y3] for left eyebrow (inner, middle, outer point displacements)
’eyebrow_right’ (list): 6-element list [x1, y1, x2, y2, x3, y3] for right eyebrow (inner, middle, outer point displacements)
’mouth_upper’ (list): 8-element list [x1, y1, x2, y2, x3, y3, x4, y4] for upper lip (left corner, left control, right control, right corner displacements)
’mouth_lower’ (list): 8-element list [x1, y1, x2, y2, x3, y3, x4, y4] for lower lip (left corner, left control, right control, right corner displacements)
time (int, optional) – The time in milliseconds to interpolate to the new offsets. If 0 (default), offsets are applied immediately. If > 0, offsets will smoothly transition from current values to target values over the specified duration.
# Immediate application face.set_component_offsets({
‘eyebrow_left’: [0, -10, 0, -5, 0, 0], # Raise left eyebrow ‘mouth_upper’: [0, 0, 0, 5, 0, 5, 0, 0] # Smile with upper lip
})
# Smooth transition over 500ms face.set_component_offsets({
‘eyebrow_left’: [0, -10, 0, -5, 0, 0]
}, time=500)
- stop_speech()[source]
Cancels the current speech command.
If the robot is speaking, the robot will stop speaking and the visemes will stop playing. If the robot is not speaking, this method will have no effect.
- Parameters
None –
- stream_file_to_browser(tag, move_face=True)[source]
Streams an audio file directly to the browser for playback.
This method sends the audio file data to the browser via WebSocket, where it will be played automatically without going through pygame.
- Parameters
filename (str) – the name of the audio file to stream (should be in pylips_phrases directory)