AI road condition detection

General
1

Is there a way to use AI on a series of images shot by a dash-cam to detect road conditions? Like, icy, snowy, etc.

Background:

As we speak, with a crew of 5, I film a winter rally in Southern Germany, Austria and Slovenia.
There is a car driving 48 hours (and another one 1 hour) ahead of time.

I would like the idea of installing a camera in the 48-hour car, that takes pictures with certain distance interval.
My initial thought was to use it like some kind of “google street view”. That makes it easier to find interesting places to film and park our crew car.

But with A.I. I could check the road conditions ahead of time. This way I can check if it is safe to start following a rally car or wait for the next one to move to our next location.

Any ideas?

2

you can do this in swift with maschine learning at apple mac in xcode ide.
and search
You only look once (YOLO)

3

You would need a large dataset to train it on though. Lots of different shots of places with different conditions. You also have to be very careful about not introducing bias by always having certain types of weather in certain places. So it could recognise the location instead of the condition.

1 Like
4

Why not simply send the geolocation with the image and use one of the many wheater APIs to get current conditions and also a forecast?

5

Most of the roads we drive on change rapidly. And weather data is not specific for the roads of the forests we hit.
Also, with that dash cam we can easily spot the locations where we can park our car safely beside the road.

6

Theoretically yes, it can be done. In practice, what you want to achieve is very difficult.

As already mentioned by Ian, you would need an enormous amount of images to train your AI. Also, the images must be good enough to capture some difference between the road conditions you are insterested in, and those differences must be diatinguishable from other differences the roads may have due to any other factor.

I am not an expert but we use machine learning at work for several thinds, including image analysis.

Julen

7

apple (xcode) provides 2 kinds of models
1 model use many folder with each a collection of same type of images.
2 model use a image collection and a json that explain part of the image in rectangles.
somehow a csv can created too.

for both it run a training, at runtime it get a camera feed in realtime or a photo input
and return a % what it could be.
i tested it with fruits, this topic maschine learning is really cool.

it have a few projects to download
https://developer.apple.com/documentation/vision

1 Like
8

@Edwin_van_den_Akker,

There are two elements to your mission:

Reconnaissance:

In my experience:

48 hours is an unreliable window.
1 hour is more reliable but not infallible.
Live feeds provide superior intelligence.

Unless you are off-road, you have at your disposal an incredible number of live feed “traffic cams” that you can surveil manually or autonomously. For example in Austria ASFINAG Webcams | ASFINAG . If you are off-road you can use a combo of town/place and traffic cam info.

You can easily map available web cams to your rally route, across Germany, Austria and Slovenia and observe them live as the rally unfolds.

Also many traffic cam networks have live alerts, you can subscribe to. Accident warning, congestion, flooding etc.

Intelligence, Decision Support

Armed the dates of and route of your rally, you can usually request via API, historical data for the last few years for those cameras you wish to target, or simply start sampling currently live feeds now. The weather regularly cycles through good, bad and ugly weather events.

You can use your AI library to machine learn common conditions per camera.

Classifying the conditions into actionable categories:

  • status red (danger), possible actions - pause/stop the rally, re-route the rally
  • status yellow (caution) possible actions - impose speed limit, impose mandatory rest breaks, require special equipment (chains etc).
  • status green (clear) possible actions - proceed as scheduled

The events insurance would still likely require the advanced rider/car proceed the actual rallygoers.

Don’t rely on AI in year 1 of using it, but rather use it in addition to observational decision support.

AI is likely overkill given what is already at your fingertips but a great fun exercise nonetheless.

Kind regards, Andrew

9

Thank you all for all the feedback. :partying_face: :pray:

I understand that using AI is a huge challenge, due to the huge amount of images I need to provide.
I should have had a dashcam installed in our car to provide those images. We drove about 2500 km (1550 miles). That should have given me a lot of images to work with.

Using weather data would not be accurate enough. Since I want to know which curve will guarantee cars sliding, and give me more spectacular shots.

Instead of IA, I think I can think of something else. I used a Raspberry Pi before, that I was able to connect with the car’s CAN-bus, via the ODB-2 port. I was able to read the car’s performance and see what the car’s anticipation to road conditions. When the cars we use starts sliding, it will activate ABS and what not. I can plot that in a graph. I can add that as meta data to a (time-lapse) video, where I can see the road as video, and the car’s behavior in a graph underneath.
This will give me a horizontal timeline I can scroll through, to find interesting places to park our media vehicle.

Maybe, I can use the car’s data for the machine learning in the future. But this will do for now.
The winter-trial is over now. We had lots of fun. It was a tough week. But next year I will do it again, for sure! So, I will have to wait another year for a Xojo companion app.

IMG_2930
IMG_29301439×1800 337 KB

1 Like
10

Not sure I follow?
The vehicles BUS is only as smart as the data it was loaded with and AI is only as smart as what it is being feed
For up the a certain time point the best bet would be some sort of weather API ?

I’m looking from this from the outside as I have not ventured into any of this instant weather warnings

11

Well, a weather API won’t provide data at specific coordinates.
I use that data to know if it could be slippery out there, due to rain prior to freezing. Or melting snow and freezing later.

The kind of cars we use provide information about what the car is doing. I tested that before. So, I can use that info when something is going on with the road. During the rally, a modern car is checking the route 48 hours, and 1 hour prior to a “race”, to see if the road is still safe enough for the participants. I can give that car a Raspberry Pi that feeds my system with the info I need.
Other cars will drive between the participants (like I was with my media car: BMW X3). I can have an identical Pi in other cars that will enrich the system’s data.

And yes, you are right about AI that needs to be trained. And that takes time… a lot of time… and a lot of images and user feedback. So, I think it is easier to see how the car is anticipating on the road.

1 Like
12

@Edwin_van_den_Akker
Maybe in time but I just do not think AI is at this level currently

13

Exactly. So, reading the car’s CAN data will work for now.
In a way it is good that some things needs to be done manually. That keeps us more in touch with reality :sunglasses:

14

@Edwin_van_den_Akker
:+1:
Hey gotten do what you got to do

1 Like
15

@Edwin_van_den_Akker, in the last photo the white Porsche was following the instructions of the AI. Great photos, looks like a wonderful event.

1 Like
16

Haha! :laughing:

Fortunately, the Porsche survived the crash. A Volvo, and several other cars, weren’t that lucky. too bad we weren’t around to see and film that.
It was an amazing event. Next year we might do the event in Norway again. Roads are even more slippery out there! :smiling_imp:

1 Like
17

I made a cute little Xojo app that told us at what location we could meet up with a certain car.
The organization liked it. Now the want me to re-build their locked-down app they use for their marshals. And I am trying to get it on Apple Carplay as well, so we can track the participants while navigating.

So, Xojo is entering the Rally realm!

Marshal App - Mock up copy
Marshal App - Mock up copy2048×2732 391 KB

3 Likes