OCP4 Tips P.07/Install OpenShift on VMware Using IPI Method and static IPs.

Hello All,

Today we are going to talk about new feature on OpenShift 4.14 enable us to install OpenShift on VMware using IPI Method and static IPs not DHCP.

to understand this article, you should have the knowledge on IPI installation method as I'm not going to dig deep on how it work or what are the alternatives to.

put in a nutshell. IPI is a fully automated way to install Openshift on many platforms which means it is responsible for creating the infrastructure and all requirements for OpenShift to be up and running.

here are the things that IPI build by itself on VMware:

• OCP nodes
• Create and attaches disks to the nodes.
• attaches the nodes to network.

and now it can also do the following:

• give node static IPs and gateway.
• give node the dns configuration.

before that we used to must have DHCP to give the created node the IPs, gateway and DNS in installation and scale out the cluster.

Steps

• create linux/bastion machine
• on the bastion machine generate ssh key

ssh-keygen -t ecdsa
cat /root/.ssh/id_ecdsa.pub        

• wget vcenter certs and make bastion machine trust them

wget https://<vcenter-ip>/certs/download.zip --no-check-certificate
unzip download.zip
mv certs/lin/* /etc/pki/ca-trust/source/anchors/
update-ca-trust
update-crypto-policies --set DEFAULT        

• create user on VMware with mentioned Privileges
• create install-config.yaml file and copy/save it befor run installer as it will destroy it.

apiVersion: v1
baseDomain: <example.com>
# proxy section is optional use it only if you are connected to internet using proxy 
proxy:
  httpProxy: https://<ip>:<port>
  httpsProxy: https://<ip>:<port>
  noProxy: <.example.com>
# this section is also optional , use it to add any crt you need your oprnshift to trust , for example your proxy crt
additionalTrustBundle: |
     -----BEGIN CERTIFICATE-----
     <your-crt>
     -----END CERTIFICATE-----
compute:
- architecture: amd64
  hyperthreading: Enabled
  name: worker
  platform:
    vsphere:
      cpus: 8
      coresPerSocket: 4
      memoryMB: 16384
      osDisk:
        diskSizeGB: 150
  replicas: 3
controlPlane:
  architecture: amd64
  hyperthreading: Enabled
  name: master
  platform:
    vsphere:
      cpus: 8
      coresPerSocket: 4
      memoryMB: 16384
      osDisk:
        diskSizeGB: 150
  replicas: 3
platform:
  vsphere:
    cluster: <cluster-name>
    datacenter: <datacenter-name>
# you can't use datastore which part of datastorage cluster, you can only use normal datastore
    defaultDatastore:  <datastore-name>
    folder: <folder-path>
# if network name contain / replace it with %2f
    network: <network-name>
    password: <password>
    username: <username>
    vCenter: <vcenter-ip>
    apiVIPs:
      - <api-vip>
    ingressVIPs:
      - <ingress-vip>
    hosts:
    - role: bootstrap
      networkDevice:
        ipAddrs:
        - <ip>
        gateway: <gw>
        nameservers:
        - <dns1>
        - <dns2>
    - role: control-plane
      networkDevice:
        ipAddrs:
        - <ip>
        gateway: <gw>
        nameservers:
        - <dns1>
        - <dns2>
    - role: control-plane
      networkDevice:
      ipAddrs:
        - <ip>
        gateway: <gw>
        nameservers:
        - <dns1>
        - <dns2>
    - role: control-plane
      networkDevice:
        ipAddrs:
        - <ip>
        gateway: <gw>
        nameservers:
        - <dns1>
        - <dns2>
    - role: compute
      networkDevice:
        ipAddrs:
        ipAddrs:
        - <ip>
        gateway: <gw>
        nameservers:
        - <dns1>
        - <dns2>
    - role: compute
      networkDevice:
        ipAddrs:
        - <ip>
        gateway: <gw>
        nameservers:
        - <dns1>
        - <dns2>   
    - role: compute
      networkDevice:
        ipAddrs:
        - <ip>
        gateway: <gw>
        nameservers:
        - <dns1>
        - <dns2>
metadata:
  creationTimestamp: null
  name: <cluster-name>
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  machineNetwork:
  - cidr: <node-subnet>
  networkType: OpenShiftSDN
  serviceNetwork:
  - 172.30.0.0/16
publish: External
fips: true
# you must enable this to use the static IP feature as it is not ready for production yet
# also not that enable it in testing cluster as it will block cluster upgrade between minor versions
# so upgrade 4.14 to 4.15 will be blocked
featureSet: TechPreviewNoUpgrade
pullSecret: <pull-secret>
sshKey: <ssh-key>
        

• run installer

openshift-install create cluster --dir=./<install-config-dir>/ --log-level=debug
```        

• wait bootstrap to complete

ssh core@bootsrap-ip

systemctl list-units --type=service
systemctl status bootkube
journalctl -u bootkube
journalctl -u -f bootkube
journalctl -u bootkube -f
#once completed exit and run 

openshift-install wait-for bootstrap-complete --dir=./<install-config-dir>/ --log-level=debug        

• wait installation to complete

export KUBECONFIG=/root/ocp-acm/install-assets/auth/kubeconfig
oc completion bash >> /etc/bash_completion.d/oc_completion

openshift-install wait-for install-complete --dir=./<install-config-dir>/ --log-level=debug        

now you have cluster provisioned on VMware using IPI and static IPs not DHCP.

let's create more nodes now.

Steps

• get cluster id

oc get machines.machine.openshift.io -n openshift-machine-api
NAME                 PHASE     TYPE   REGION   ZONE   AGE
ocp-ygtrf-master-0   Running                          28d
ocp-ygtrf-master-1   Running                          28d
ocp-ygtrf-master-2   Running                          28d
ocp-ygtrf-worker-0   Running                          28d
ocp-ygtrf-worker-1   Running                          28d
ocp-ygtrf-worker-2   Running                          28d

# cluster if form is <cluster-name>.<random-charachters>
# in our case it is ocp-ygtrf        

• get any worker machine created by the installer and save it as yaml to copy required info from it if needed

oc get machines.machine.openshift.io ocp-ksvrj-worker-0 -oyaml -n openshift-machine-api        

• create below new worker static file

apiVersion: machine.openshift.io/v1beta1
kind: Machine
metadata:
  creationTimestamp: null
  labels:
    machine.openshift.io/cluster-api-cluster: ocp-ygtrf
    machine.openshift.io/cluster-api-machine-role: worker
    machine.openshift.io/cluster-api-machine-type: worker
    node-role.kubernetes.io/worker: ""
    node-role.kubernetes.io: worker
  name: <cluster-id>-worker-<number>
  namespace: openshift-machine-api
spec:
  lifecycleHooks: {}
  metadata: {}
  providerSpec:
    value:
      apiVersion: machine.openshift.io/v1beta1
      credentialsSecret:
        name: vsphere-cloud-credentials
      diskGiB: 150
      kind: VSphereMachineProviderSpec
      memoryMiB: 32768
      metadata:
        creationTimestamp: null
      network:
        devices:
        - gateway: <gw>
          ipAddrs:
          - <ip/24>
          nameservers:
          - <dns1>
          - <dns2>
          networkName: <network-name>
      numCPUs: 8
      numCoresPerSocket: 4
      snapshot: ""
      template: <template>
      userDataSecret:
        name: worker-user-data
      workspace:
        datacenter: <datacenter>
        datastore: </datacenter/datastore/datastore>
        folder: </datacenter/vm/folder>
        resourcepool: </datacenter/host/cluster/Resources>
        server: <vcenter>
        

and finally the magic happened once applied the file, new worker node we will be created with static configured IP.

Hope you have enjoyed reading with about that new feature in OPC 4.14.

all examples in this article tested on OCP 4.14.10 and VMware 7.

for more info about OCP IPI Installation OCP_IPI_VMware

for more info about IPI on VMware using Static IP OCP_IPI_VMware_Static_IP

If you need to know anything about Openshift please feel free to contact me and read my past articles maybe it helps.

Subscribe on LinkedIn https://www.dhirubhai.net/build-relation/newsletter-follow?entityUrn=6909177885404622848