How Tesla Vision Sees Through Inclement Weather

Tesla’s camera-only approach to autonomy is frequently criticized for its behavior in harsh weather. Skeptics argue that radar, LiDAR, and ultrasonic sensors (USS) are better suited for poor conditions, while Tesla maintains that Tesla Vision performs better than multiple sensors.

This month, the company published a patent titled "Fail-Safe Corrective Actions Based on Vision Information for Autonomous Vehicles," describing how FSD responds to adverse weather. Instead of halting when visibility worsens, the system identifies the type of obstruction and applies countermeasures to continue operating safely.

Visibility Grid and Scene Tags

To handle degraded camera input, Tesla’s neural networks continuously process frames from the vehicle’s cameras at high rates—about 30 Hz (HW3) or 60 Hz (AI4). When a camera’s view is obstructed, FSD doesn’t simply flag an error. It divides the image into a grid of rectangular pixel regions and assigns each region a visibility value.

The scale ranges from 0 to 3, where 0 indicates a clear view and 3 means the image is fully occluded. After mapping the severity, the system goes further by identifying the cause. The neural network assigns specific "scene tags" to the affected regions.

It can tag conditions such as haze, condensation, ice, water, rain, sun glare, fog, smoke, and tire spray. The system can also detect a dirty windshield or hardware issues like dead pixels.

Active Hardware Countermeasures

The patent’s notable element is how the vehicle reacts to these tags. Tesla integrated the visibility neural network with vehicle hardware so that a high visibility value paired with a specific tag triggers automated, fail-safe actions. If the scene is tagged as rainy, the system can automatically activate the windshield wipers, while other low-visibility tags can turn on the headlights or hazard lights.

If it detects "snow" or "condensation," the vehicle can engage the windshield heaters or air-conditioning defoggers to clear the glass. The neural networks can also change lanes to avoid hazards, such as moving away from heavy tire spray from a leading truck. Among the features described, this appears to be the only one not implemented in FSD today.

During rain FSD 14.3 locks you into chill or sloth. Standard, hurry, and mad Max are not available. pic.twitter.com/hm1G4ibgla

— Dirty Tesla (@DirtyTesLa) April 10, 2026

Reducing Phantom Braking

Beyond clearing camera views, the patent addresses phantom braking. Vision systems have historically produced false positives when visibility is low; for example, a dense fog bank or an exhaust plume can be mistaken for an object.

The patent outlines a method to cut down on false detections. Because the neural network labels scenes with visibility values, it can recognize when it’s operating in a compromised environment.

If the car detects what looks like an obstacle while the visibility grid indicates severe occlusion from smoke or fog, FSD can infer that the object is likely a visual artifact. Using this visibility context, the vision-based model suppresses unnecessary braking, resulting in a smoother, more confident drive.