World’s first native color LiDAR gives machines human-like vision

[hist78]

"For years, machines have navigated the world color-blind. LiDAR sensors – the laser-based eyes of self-driving cars, industrial robots, and inspection drones – build precise 3D maps of their surroundings, but everything is built of monochrome geometric shapes. Ouster's new Rev8 sensor family aims to change that, not by bolting a camera onto a LiDAR unit, but by fusing color directly into every point of data the sensor captures."

"The flagship model of the Rev8 family is the OS1 Max, a 256-channel sensor with a detection range of up to 200 m (656 ft) at 10% reflectivity – meaning it can spot surfaces that absorb most of the light hitting them – and up to 500 m (1,640 ft) under optimal conditions. Its field of view spans 45 degrees vertically and 360 degrees horizontally, and Ouster claims it doubles both the range and resolution of its previous generation, the Rev7."

World’s first native color LiDAR gives machines human-like vision

For years, machines have navigated the world color-blind. LiDAR sensors – the laser-based eyes of self-driving cars, industrial robots, and inspection drones – build precise 3D maps of their surroundings, but everything is built of monochrome geometric shapes. Ouster's new Rev8 sensor family aims…

newatlas.com

 

[Xebec]

"For years, machines have navigated the world color-blind".

I guess in this timeline, color cameras haven't been invented yet?

(not a poke at you hist78 - just knocking the marketing of this Lidar company )

 

[hist78]

"For years, machines have navigated the world color-blind".

I guess in this timeline, color cameras haven't been invented yet?

(not a poke at you hist78 - just knocking the marketing of this Lidar company )

Cameras have difficulties under heavy snow, rain, or fog for FSD. They are also challenged by direct sunlight conditions, such as during sunrise or sundown. LiDAR with color capability can greatly improve the accuracy and safety of FSD.

 

[Xebec]

Cameras have difficulties under heavy snow, rain, or fog for FSD. They are also challenged by direct sunlight conditions, such as during sunrise or sundown. LiDAR with color capability can greatly improve the accuracy and safety of FSD.

FWIW, it's not as clean as a scenario as it sounds -- Lidar also has issues in these scenarios, too:

Helping automated vehicles ‘see’ through fog and snow | Center for Transportation Studies

While fully automated vehicles (AVs) have not yet reached the roads, the work

www.cts.umn.edu

“While lidar works well on clear days, it can be unreliable in foggy, rainy, or snowy weather—creating a potential hazard for all road users,” says Will Northrop, professor in the Department of Mechanical Engineering and the project’s principal investigator.

..

The challenge with Lidar + Camera Vision is, as the amount of input increases, the compute power required to 'resolve' and drive the car properly also increases substantially (perhaps more than 2X because of the need to combine and deconflict data).

I agree more sensors should mean more safety, but 6-8 cameras + software that doesn't get distracted should be able to beat human eyes and reaction times for overall safety - especially given cameras can be coated and tuned to see better in low light and snow conditions than human eyes through a windshield).

 

[hist78]

FWIW, it's not as clean as a scenario as it sounds -- Lidar also has issues in these scenarios, too:

Helping automated vehicles ‘see’ through fog and snow | Center for Transportation Studies

While fully automated vehicles (AVs) have not yet reached the roads, the work

www.cts.umn.edu

“While lidar works well on clear days, it can be unreliable in foggy, rainy, or snowy weather—creating a potential hazard for all road users,” says Will Northrop, professor in the Department of Mechanical Engineering and the project’s principal investigator.

..

The challenge with Lidar + Camera Vision is, as the amount of input increases, the compute power required to 'resolve' and drive the car properly also increases substantially (perhaps more than 2X because of the need to combine and deconflict data).

I agree more sensors should mean more safety, but 6-8 cameras + software that doesn't get distracted should be able to beat human eyes and reaction times for overall safety - especially given cameras can be coated and tuned to see better in low light and snow conditions than human eyes through a windshield).

It was published by a research team at the University of Minnesota in August 2022 (the research itself was probably completed in March 2022). Four years later, I’m wondering whether the analysis would be different given the advances in technology.

Original University of Minnesota research paper:

Can Automated Vehicles "See" in Minnesota? Ambient Particle Effects on LiDAR

Can Automated Vehicles "See" in Minnesota? Ambient Particle Effects on LiDAR

This project will use a combination of laboratory experimentation and road demonstrations to better understand the reduction of LiDAR signal and object detection capability under adverse weather conditions found in Minnesota. It will also lead to concepts to improve LiDAR systems to adapt to...

conservancy.umn.edu

 

[Xebec]

It was published by a research team at the University of Minnesota in August 2022 (the research itself was probably completed in March 2022). Four years later, I’m wondering whether the analysis would be different given the advances in technology.

Certainly possible, and I suspect Color LiDAR will have a lot of interesitng uses too outside automobile AI. IIRC, warfare drones are heavily dependent upon LiDAR - and these drones also do things we would find useful as civilians.

..

One related item, re: the engineering costs with LiDAR+Vision vs Vision only:

A Tesla FSD computer +camera setup consumes about 150W during driving, while Waymo's Camera+LiDAR+computer suite consumes around 1,000W. (Rumored 3kW at one point). The increased energy comes from the additional compute required to reconcile everything, the LiDAR hardware, and likely an older chip process for the compute hardware than what Tesla uses.

In Robotaxi use terms (~ 20-30 mph average speed), the LiDAR+Camera setup vs. Camera only means:

+ ~ $3-5/day more energy costs
+ ~ $3,000-4,000+ additional upfront cost for the LiDAR hardware including the sensor(s), cabling, mounting externally, etc. (google search guessing)
+ Increased downtime for the car to charge more often (~ 30-40 miles/day of driving energy spent more than Camera only)
+ Additional cooling and engineering required to deal with 1,000W vs 150W (at some cost)

.. Spitballing, a difference of about 4 cents per mile of Robotaxi service over the lifetime.