I recently found that the OsmAnd app lets you adjust the safety of your bike routes, so you can prefer safety over distance.
When navigating, click the "Ride Style" button and choose "prefer unpaved roads." That name made me think it would find gravel/off-road trails, but it actually selects safer roads. In my experience this setting chooses the optimal routes--it's finding the same general path that I would pick based on local knowledge, and it found improvements where I could take a slightly different street for a few blocks to avoid cars!
Also, OsmAnd~ is available via Fdroid with all the paywalls removed.
Sounds like it might vary. I don't have dirt roads near me, so it selects safe roads with slower traffic instead. Hopefully they can add a proper "prefer safer roads" option, but this works for me in the meantime.