With current "Ceph production example" the difference between ceph's
public and storage network is not clear.
We assign Storage Network to compute nodes, but it's not used there.
We also asign Storage Network to ceph monitors, but it's not used there
as well.
Same problems apply to AIO environment.
As Dmitriy suggested in [1], ceph should not use mgmt network for
storage traffic.
This change makes ceph use storage network for:
- OSD<>OSD communication
- client<>OSD communication
- client<>MON communication
I think it's the most common scenario where all ceph-related traffic
uses dedicated(storage) network and do not depend on mgmt network.
This change affects both "Ceph production example" docs and AIO
environments.
[1] https://review.opendev.org/c/openstack/openstack-ansible/+/856566
Change-Id: I74387a2e961e2b8355ea6a0c889b2f5674233ebf
bb1287555c