Preferences

1 point
0 comments trzy github.com
A robot that uses an iPhone Pro for its compute and sensor stack, controlled entirely by Claude 3.5-Sonnet or GPT-4. The mobile base is a salvaged hoverboard with electronics replaced.

Mobile phones are incredibly capable -- loads of compute (with neural network acceleration), great connectivity (LTE, BLE, WiFi), front and rear-facing cameras, LiDAR, microphones, a screen and speakers built-in for output. AR frameworks provide very sophisticated perception capabilities out of the box that are incredibly useful for navigation. RoBart uses ARKit's scene meshing to identify navigable areas and communicates that to the LLM by annotating camera images with numbered landmarks.

I'd love to see more projects leveraging phones for robotics prototypes, as I've found the development cycle to be much more pleasant than working with something like ROS and a Jetson board.


This item has no comments currently.

Keyboard Shortcuts

Story Lists

j
Next story
k
Previous story
Shift+j
Last story
Shift+k
First story
o Enter
Go to story URL
c
Go to comments
u
Go to author

Navigation

Shift+t
Go to top stories
Shift+n
Go to new stories
Shift+b
Go to best stories
Shift+a
Go to Ask HN
Shift+s
Go to Show HN

Miscellaneous

?
Show this modal