Curiosity, Passion, and Learning to Learn

SJSU CyberAI Summer Camp · June 22, 2026

Curiosity, Passion,
and Learning to Learn

Matthias Müller
A journey from Lego to OpenBot.

The Foundation

Logic meets the physical world

Lego Mindstorms was my first real introduction to robotics. The realisation that logic could control physical things changed everything.

The Foundation

Taking things apart

I've been fixing electronics since I was a kid — opening things up just to see how they worked. That instinct was the seed of everything.

The Skills Behind the Projects

Every project I built was preparing me for the next.

The skills didn't come from a curriculum.
They compounded, one project at a time.

Lego Mindstorms

→

Electronics / Tinkering

→

Soldering

→

3D Printing

→

Custom Drones

The Expected Path

Meanwhile, I followed the expected path.

🎓

B.Sc. Electrical Engineering

The degree everyone said to get.

💍

Got Married

The milestone everyone expected next.

💼

Started Working in Industry

A real job at BMW. Stable. Predictable.

Nothing wrong with that.
But the curiosity didn't stop.
The side projects were just getting started.

BMW

The dream job.

The Work

Developing mild-hybrid alternators — now in most cars on the road. Test driving early prototypes like the BMW i8.

The Catch

But the basement called every weekend.

Starting Over

I quit.

BMW felt like the void. So I left — driven by instinct, not comfort. Managing suppliers wasn't why I studied engineering.

Decision 1

Left BMW

Desiring hands-on work over project management.

Then I started over.

↓

Decision 2

MS in EE Communications

Theoretical. Not applied. Still not quite right.

Started a Master's in Electrical Engineering, focused on communications theory. But it was not a great fit, and the next decision only took a few months.

↓

Decision 3

Changed Topic & Adviser

Computer Vision. Building. Following the passion.

PhD

Computer vision for drones.

Object detection, tracking, drone autonomy.
Hobby and research were converging on the same thing: getting algorithms to work on real hardware.

First autonomous driving project

A 1:5 scale self-driving RC truck.

Built during a research internship — retrofit a 1:5 scale RC truck with cameras and sensors, then teach it to drive itself.

This is where everything starts.

ICRA 2018

Conditional Imitation Learning

CIL architecture: RGB image + speed measurement + planner command → neural network → steering and pedals

We taught this truck to drive using human demonstrations.

Existing wheeled robots

All too expensive to scale.

F1/10 · $3,600

MuSHR · $900

DJI Robomaster · $500

Duckiebot · $280

OpenBot

Smartphone brain + off-the-shelf parts

$50

Mechanical design

3D-printed body.

Prints in less than a day · $5 in PLA

Electrical design

Arduino · motor driver · batteries · sensors.

Wiring

Optional PCB

A PCB collapses the wire mess into one board you plug into — same circuit, no soldering jungle.

Hardware · Assembly

Build it in 15 minutes.

Off-the-shelf electronics · 3D-printed body · a screwdriver.

Bill of materials

Fifty bucks? A whole robot?

3D-printed body$5

Arduino Nano$8

4 motors + tires$14

Motor driver$3

3× 18650 batteries$15

Sonar + wheel encoders$5

Total

$50

($30 in bulk)

…and the brain you already have in your pocket.

The brain

Your phone is a robot computer.

📷

HD camera

🧠

CPU + GPU + AI chip

🧭

IMU + GPS

📶

WiFi · BT · 4G/5G

🔊 🎤 🖥

Speaker + mic + display

🔋

Battery + charging

Software stack

Smartphone runs the AI.
Microcontroller drives the motors.

📱 Smartphone · Android app

📷 Camera + IMU + GPS

🧠 Neural net (TF Lite)

📊 Sensor logger

🔊 Voice + display

⇄

USB

— or —

BLUETOOTH

⚙️ Robot body · MCU

🔧 PWM motor control

🛞 Wheel encoders

📡 Sonar

🔋 Battery monitor

Beyond the phone

Three ways to drive and train it.

🐍 Python SDK

Control the robot from Python

🌐 Policy training app

Train driving policies in-browser

📡 Live teleop

Web app · live video · Node.js backend

AI App #1

Person following.

📷

Camera frame

→

SSD + MobileNet

tiny detector designed for phones
trained on COCO · 90 classes

→

🎯

Bounding box
around the person

Pick the target

Only consider person detections above 50% confidence. If there are multiple people, follow the one with the highest score.

Visual servoing

Steer so the center of the bounding box stays in the middle of the frame. Speed up when the box gets small, slow down when it gets big.

Runs in real time on any modern smartphone — quantized model, CPU-only, no cloud.

AI App #2

Autonomous navigation.

Similar to Tesla Autopilot — an end-to-end neural network that drives the car autonomously.

Conditional Imitation Learning — recap from earlier

We took that — and shrank it to fit on a phone.

Autonomous nav · the model

Same accuracy as the big nets.
7× smaller.

PilotNet · 9.6M params

92% · 83%

distance · success

CIL · 10.7M params

94% · 89%

distance · success

Ours · 1.3M params

94% · 89%

distance · success

⚡ Runs faster than real-time on pretty much any modern phone.

Coffee Racer

Autonomous coffee delivery.

OpenBot-Fleet · ICRA 2024

From 1 robot to 72 homes.

If we want robots to work in your house, we can't train in one lab.

Hardware V2

Injection-molded · plug-and-play.

DIY was great for makers. 72 robots need real production · same cost, more durable.

The cloud loop

Robots collect data.
Policy updates in the cloud.
Fleet gets the new policy.

72 robots

in real homes

⇄

☁️

Firebase

storage · sync

⇄

🧠

RL training

TF-Agents · SAC

15,642 recordings · 1M+ frames · self-supervised U-Net floor segmentation.

Fleet results

Diversity beats data quantity.

Zero-shot · unseen homes

Best baseline15%

Ours60%

+ <200 finetune episodes82.5%

Diversity ablation · success rate

1 env50%

6 envs70%

72 envs85%

Same data per env. 1 home → 50%. 72 homes → 85%. Where the data comes from matters more than how much.

Impact

Side projects → published research.

ICRA 2018 1,669 citations

End-to-end Driving via Conditional Imitation Learning

Codevilla, Müller, López, Koltun, Dosovitskiy

CoRL 2018 324 citations

Driving Policy Transfer via Modularity and Abstraction

Müller, Dosovitskiy, Ghanem, Koltun

ICRA 2021 ⭐ 1,818 · 🍴 300

OpenBot: Turning Smartphones into Robots

Müller, Koltun

ICRA 2024 72 robots · 1M+ frames

OpenBot-Fleet: A System for Collective Learning with Real Robots

Müller, Brahmbhatt, Deka, Leboutet, Hafner, Koltun

Community

👥 …and then the community started building.

Community · MTV

A Mars-rover-style OpenBot.

Multi-Terrain Vehicle · 6 wheels · rocker-bogie suspension · climbs stairs.

Community · OpenBot Yoda

Real-life Baby Yoda with AI.

By Manuel Ahumada · because open source.

OpenBot for education

Autonomous driving on a track.

Students in Korea and Singapore training nav policies on the same body you'd build for $50.

🇰🇷 Korea

🇸🇬 Singapore

📺 Full instructional + playground tutorial playlists on the OpenBot YouTube channel.

OpenBot for education

Used in classrooms worldwide.

🇩🇪 Germany

🇮🇳 India

🇵🇹 Portugal

🇸🇬 Singapore

🇰🇷 South Korea

🇬🇧 UK

🇰🇬 Kyrgyzstan

…and more

OpenBot for education

Lots of videos on YouTube.

🎓 How to train your robot

📺 Lesson 04 · Motion

Everything's open source

openbot.org

📐 Hardware designs 💾 Code 📱 Apps 🎬 Videos 📝 Papers

Beyond OpenBot

Three things this taught me.

🔭

Follow curiosity

🛠

Build > consume

🤝

Use AI as a tool

Reality check

AI is in everyone's workflow now.

🎓

Students

Homework. Essays. Research. Lecture notes.

💻

Engineers

Copilot · Cursor · Claude Code. Most new code is AI-assisted.

✍️

Writers

Drafts. Edits. Brainstorms. Translations.

🔬

Researchers

Literature reviews. Data analysis. Code prototypes.

The question isn't if you use AI. It's how.

AI as Tool, Not Crutch

The AI interaction spectrum.

Do It Yourself

Roll up your sleeves.
No AI.

✓ Learning, edge cases, high stakes

⚠ Autopiloting through the hard parts

Debate

Stress-test your view
with AI.

✓ After forming your own view

⚠ Before you've thought at all

Delegate

Hand off the work
that won't sharpen you.

✓ Mechanical, repetitive work

⚠ Work that sharpens you

Don't default to delegate.
AI as tutor extends you. AI as crutch replaces you.

Stop Consuming. Start Building.

You don't prepare for the opportunity.
Projects are the preparation.

Build

Build what excites you.
Ignore prestige.

Stretch

Aim beyond your current skill.
That's where growth lives.

Compound

Pick projects that
unlock the next.

Curiosity today. Opportunity tomorrow.
Build now. Be ready always.

The Builder's Resource Checklist

Tools worth your time.

Paul Graham

Great essays, for example "How to Do Great Work" and "How to Do What You Love".

Khan Academy

Free courses on math, science, CS — at your own pace.

Claude Code

Automate tedious syntax so you can focus on architecture and the "Why".

NotebookLM

Synthesise and learn complex new domains quickly. Scaffold, don't shortcut.

Final Thought

"The way to do great work
is to love what you do."

— Steve Jobs, Stanford Commencement, 2005

Get Ready. Start Now.

Get ready.

Follow your curiosity and passion. When the big opportunity comes, you won't have time to prepare — you need to be ready, and you are, because you've been building for years. Don't wait for a perfect plan.

Start now.

One More Thing

This presentation
was a project.

Before

PowerPoint. Keynote. Google Slides.

This time

Static HTML. Pure code. No slides app.
Built with Claude Code.

View online

Scan to open this presentation.

Stay curious. Find interesting projects. Never stop learning.