Teleoperation
Details about running the teleoperation flow
Requirements
- linuks-kbot v0.0.4+
- kos-kbot 0.1.3+
Teleop
Starting WebRTC stream
docker run --rm -it --network=host \
--device=/dev/video45 \
--add-host xr.kscale.ai:10.33.20.104 \
--add-host host.docker.internal:127.0.0.1 \
ghcr.io/kscalelabs/gst-rs-webrtc:latest \
gst-launch-1.0 \
webrtcsink name=ws \
meta="meta,name=kbot-$(cat /sys/class/net/wlan0/address | tr -d ':')" \
enable-control-data-channel=true \
signaller::uri="wss://xr.kscale.ai:8585" \
v4l2src device=/dev/video45 io-mode=2 ! \
video/x-raw,format=YUY2,width=1280,height=1080,framerate=30/1 ! \
videoconvert ! \
videoflip method=vertical-flip ! \
video/x-raw,format=I420 ! ws.
Teleop script
To start the teleop data channel over WebRTC, see the script here .
VR Headset link
View the VR headset teleoperation portal here
Data collection
Starting a recording
rec_uuid = kos.process_manager.start_kclip("pick up the blue cup")
Stopping a recording
rec_uuid = kos.process_manager.stop_kclip()
Exploring krec
import krec
rec = krec.extract_from_video('recording_20250319_193701_real_test_1_9c85a9a4-f5b4-437c-9425-d0a2ec732570.krec.mkv')
>>> rec
KRec(frames=1337, header=KRecHeader(uuid='9c85a9a4-f5b4-437c-9425-d0a2ec732570', task='real test 1', robot_platform='KBot', robot_serial='00000000', configs=0))
>>> print(rec.display_frame(300))
Frame 300
=========
Video timestamp: 8919177055
Video frame number: 108
Inference step: 4739
Actuator States (10)
---------------
ID 11: online=true, pos=Some(89.11151123046875), vel=Some(-2.9900929927825928), torque=Some(4.183113098144531), temp=Some(24.0), volt=None, curr=None
ID 12: online=true, pos=Some(16.007286071777344), vel=Some(-1.101572871208191), torque=Some(-0.4971389770507813), temp=Some(24.0), volt=None, curr=None
ID 13: online=true, pos=Some(-22.379470825195313), vel=Some(-71.89730072021484), torque=Some(0.8313884735107422), temp=Some(24.0), volt=None, curr=None
ID 14: online=true, pos=Some(-65.06756591796875), vel=Some(-8.578498840332031), torque=Some(-0.8412456512451172), temp=Some(28.0), volt=None, curr=None
ID 15: online=true, pos=Some(-6.245880126953125), vel=Some(-3.038723945617676), torque=Some(0.0313873291015625), temp=Some(26.0), volt=None, curr=None
ID 21: online=true, pos=Some(0.8899463415145874), vel=Some(0.6119849681854248), torque=Some(0.2481117248535156), temp=Some(22.0), volt=None, curr=None
ID 22: online=true, pos=Some(-2.4664306640625), vel=Some(6.067502975463867), torque=Some(-0.022891998291015625), temp=Some(22.0), volt=None, curr=None
ID 23: online=true, pos=Some(-3.241006851196289), vel=Some(-12.502197265625), torque=Some(0.20570755004882813), temp=Some(25.0), volt=None, curr=None
ID 24: online=true, pos=Some(3.8343043327331543), vel=Some(1.500018835067749), torque=Some(-0.0002593994140625), temp=Some(23.0), volt=None, curr=None
ID 25: online=true, pos=Some(6.383166790008545), vel=Some(-2.730982780456543), torque=Some(-0.2679634094238281), temp=Some(26.0), volt=None, curr=None
Actuator Commands
----------------
ID 11: pos=90.703125, vel=0, torque=0
ID 12: pos=15.908203, vel=0, torque=0
ID 13: pos=-25.751953, vel=0, torque=0
ID 14: pos=-66.26953, vel=0, torque=0
ID 15: pos=-6.2402344, vel=0, torque=0
ID 21: pos=0.9667969, vel=0, torque=0
ID 22: pos=-2.4609375, vel=0, torque=0
ID 23: pos=-3.1640625, vel=0, torque=0
ID 24: pos=7.3828125, vel=0, torque=0
ID 25: pos=6.328125, vel=0, torque=0
Updated 8 days ago