LOKI OpenShift on OpenStack egress IP Address per project

LOKI OpenShift on OpenStack egress IP Address per project

A couple of weeks ago, we wrote an article about the benefits of LOKI, a combination of Linux, OpenStack, Kubernetes and network infrastructure. This article can be found here: https://www.linkedin.com/feed/update/urn:li:ugcPost:6899649383181148160?updateEntityUrn=urn%3Ali%3Afs_updateV2%3A%28urn%3Ali%3AugcPost%3A6899649383181148160%2CFEED_DETAIL%2CEMPTY%2CDEFAULT%2Cfalse%29 .

When using LOKI, in this case OpenShift (or OKD) on OpenStack you know that OpenShift egress traffic will always use the IP address of the virtual router. This is not always favorable because it provides you not much control. Systems are mostly configured separately from each other as you don’t want your entry system at the same virtual place as your client data. When you would like to have control of what projects can access other services on your network, we have a solution for you. So here, is a short case study in the form of a blog on how we solve this.
When using LOKI, in this case OpenShift (or OKD) on OpenStack you know that OpenShift egress traffic will always use the IP address of the virtual router. This is not always favorable because it provides you not much control. Systems are mostly configured separately from each other as you don’t want your entry system at the same virtual place as your client data. When you would like to have control of what projects can access other services on your network, we have a solution for you. So here a short case study in the form of a blog on how we solve this.
In this particular case we have OpenShift running on OpenStack with an external firewall. We want to separate the data from other systems to make sure the data from users are kept safe. Furthermore, we want to remain in control over the infrastructure. We looked if we could achieve this within a LOKI infrastructure; OpenStack in combination with OpenShift and open networking. According to the OpenShift manual this was possible. However, there were a lot of side notes to pay attention to.

One option would be Kuryr, with the downside that every Pod will get its own IP address from OpenStack, causing a lot of overhead. For this reason, we chose another solution.
Our solution starts with the installation of the cluster where it normally installs OVNkubernetes as the standard network type. However, this network type does not support custom egress when using OpenShift on OpenStack. We resolved this by using OpenShiftSDN instead. After some trials and tests, we eventually tried to put it on another network infrastructure layer. This gave mixed results. It was good and bad at the same time. However, we wanted it only to be good, so we kept trying. Lastly, we connected OpenShift’s machine network to the provider network of OpenStack and this gave us a successful result.

The first commando to send to the cluster, when the cluster is up and running:

$ oc patch netnamespace –type=merge -p ‘{“egressIPs”: [“ip_address>”]}’
The second commando is to give the same IP address to a host:

$ oc patch hostsubnet –type=merge -p ‘{“egressCIDRs”:[“ip_address_range_1>”, ““]}’
After this it is important to allow address pairs in OpenStack Neutron. To allow the approval for use of the IP address.

So, there you have it; in this we have created a solution that gives you more control of the network within you LOKI OpenStack OpenShift infrastructure. More information on this OpenShift subject can be found here: https://docs.openshift.com/container-platform/4.9/networking/openshift_sdn/assigning-egress-ips.html
If you have any questions about this solution or LOKI in general; we are happy to have a talk with you.