microk8sを利用7-macmini M4でarm64のkubernetesクラスタ

作成イメージ

arm64のホストをutmの中に作って⭐️印の箇所を作ってく。

だんだんデカなってきたな。頭の体操にはええか。

<----------- M4 mac -------dev->     <--------------------- VMware on Windows11 pro -------------stg-->    <-------- Google cloud ---prd-->

+-M4 macmini sequoia-----------+     +-kubelinux(ubuntu24) --------+    +-local ubuntu24 linux--------+    +-GCE ubuntu24 linux----------+
| +-utm ---------------------+ |     | +-microk8s-------+ +-vmdk-+ |    | +-docker---------+ +-vmdk-+ |    | +-docker---------+ +--pv--+ |
| |  +-sequoia arm64-------+ | |     | |  +-Pod-------+ | | data | |    | |  +-container-+ | | data | |    | |  +-container-+ | | data | |
| |  |  vscode/cyberduck   | | |     | |  | Django    | | |  d1  | |    | |  | Django    | | |  d1  | |    | |  | Django    | | |  d1  | |
| |  |  office/brew/rdp    | | |     | |  +-----------+ | +------+ |    | |  +-----------+ | +------+ |    | |  +-----------+ | +------+ |
| |  +---------------------+ | |     | |  +-Pod-------+ | |      | |    | |  +-container-+ | |      | |    | |  +-container-+ | |      | |
| |  +-kubearm(ubu24)  ⭐️--+ | |     | |  | mariadb   | | |  d2  | |    | |  | mariadb   | | |  d2  | |    | |  | mariadb   | | |  d2  | |
| |  | ctr microk8s        | | |     | |  +-----------+ | +------+ |    | |  +-----------+ | +------+ |    | |  +-----------+ | +------+ |
| |  |+-container-+        | | |     | |  +-Pod-------+ | |      | |    | |  +-container-+ | |      | |    | |  +-container-+ | |      | |
| |  ||kubernetes |        | | | <-  | |  | xrdp-ubu24| | |  d3  | | <- | |  | xrdp-ubu24| | |  d3  | | <- | |  | xrdp-ubu24| | |  d3  | |
| |  |+-----------+        | | | <-  | |  +-----------+ | +------+ | <- | |  +-----------+ | +------+ | <- | |  +-----------+ | +------+ |
| |  |+-container-+ +/data+| | | <-  | |                |          | <- | |                |          | <- | |  +-container-+ |          |
| |  ||Django     | | d1  || | |     | |                |          |    | |                |          |    | |  | gitlab    | |          |
| |  |+-----------+ +-----+| | |     | |                |          |    | |                |          |    | |  +-----------+ |          |
| |  |+-container-+ |     || | |     | |  +-Pod-------+ | +------+ |    | |  +-container-+ | +------+ |    | |  +-container-+ | +------+ |
| |  ||mariadb    | | d2  || | |     | |  | https     | | |  d4  | |    | |  | https     | | |  d4  | |    | |  | https     | | |  d4  | |
| |  |+-----------+ +-----+| | |     | |  +-----------+ | +------+ |    | |  +-----------+ | +------+ |    | |  +-----------+ | +------+ |
| |  |+-container-+ |     || | |     | +----------------+          |    | +----------------+          |    | +----------------+          |
| |  ||xrdp-ubu24 | | d3  || | |     +-----------------------------+    +-----------------------------+    +-----------------------------+
| |  |+-----------+ +-----+| | |                                                 |||
| |  |+-container-+ +-----+| | |     +--smb share -----------------+             |||
| |  ||https      | | d4  || | |     |  windows smb share          |             |||
| |  |+-----------+ +-----+| | |     +-----------------------------+             |||
| |  +---------------------+ | |                                                 |||
| +--------------------------+ |                                                 |||
|      ^ ^                     |                                                 |||
|      | |  +---------------+  |                                                 |||
|      | |  |Django & xrdp  |  |                                                 |||
|      | |  |docker image to|  |                                                 |||
|      | |  |ctr registry   |  |                                                 |||
|      | |  +---------------+  |                                                 |||
|      | |                     |                                                 |||
| +-rancher desktop----------+ |                                                 |||
| | docker on lima           | |                                                 ||| +---------------------------------+
| |  +-container-+  +------+ | |                                                 ||| | d1/d2/d3 text or binary data    |
| |  | Django    |  |  d1  | | |                                                 ||| | d4 tls data that make each host |
| |  +-----------+  +------+ | |                                                 ||| +---------------------------------+
| |  +-container-+  |      | | | <-----------------------------------------------+||
| |  | mariadb   |  |  d2  | | | <------------------------------------------------+|
| |  +-----------+  +------+ | | <-------------------------------------------------+
| |  +-container-+  |      | | |
| |  | xrdp-ubu24|  |  d3  | | |
| |  +-----------+  +------+ | |
| |  +-container-+  +------+ | |
| |  | https     |  |  d4  | | |
| |  +-----------+  +------+ | |
| +--------------------------+ |
|                              |
| +-music----------+           |
| | 8400(37GB)     |           |
| +----------------+           |
+------------------------------+

結論

microk8sでサラっと動かすつもりやったんやけど、multipassがうまいこと行かんかったからUTM使うことにした。

arm64-microk8s-kubearm

何回かやって気づいたのは、Podの削除するのが前よりも時間かかって1分ぐらい待たなアカンようになったな。

仮想マシン作成

UTMのapple virtualizationホストからはできてるんやけど、microk8sのkubernetesクラスタのPodから宅内DNSの参照できるようになるのが時間取られた。

macにmicro8sインストールはできても、multipassがアカン

microk8sのパッケージ入れた。

nari@narimac-mini ~ % brew install ubuntu/microk8s/microk8s
==> Auto-updating Homebrew...
Adjust how often this is run with HOMEBREW_AUTO_UPDATE_SECS or disable with
HOMEBREW_NO_AUTO_UPDATE. Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
==> Fetching dependencies for ubuntu/microk8s/microk8s: kubernetes-cli
==> Fetching kubernetes-cli
==> Downloading https://ghcr.io/v2/homebrew/core/kubernetes-cli/manifests/1.32.0
Already downloaded: /Users/nari/Library/Caches/Homebrew/downloads/19dbdc5c4ddcb06e2a0437e78cd551eebcb45db1bfdc8c8e0555a090d1396222--kubernetes-cli-1.32.0.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/kubernetes-cli/blobs/sha256:83f41dddaff07d9a1b81536
Already downloaded: /Users/nari/Library/Caches/Homebrew/downloads/da404229b288561fb4ccf80fd0f8246d8b1151cf198105b920c26ded5cbb341f--kubernetes-cli--1.32.0.arm64_sequoia.bottle.tar.gz
==> Fetching ubuntu/microk8s/microk8s
==> Downloading https://github.com/canonical/microk8s/archive/refs/tags/installer-v2.3.4.tar.gz
Already downloaded: /Users/nari/Library/Caches/Homebrew/downloads/281c9d82705923ca3136ff18904081c5aa47bd1c6cd754f6562810a7d68f9af0--microk8s-installer-v2.3.4.tar.gz
==> Installing microk8s from ubuntu/microk8s
==> Installing dependencies for ubuntu/microk8s/microk8s: kubernetes-cli
==> Installing ubuntu/microk8s/microk8s dependency: kubernetes-cli
==> Downloading https://ghcr.io/v2/homebrew/core/kubernetes-cli/manifests/1.32.0
Already downloaded: /Users/nari/Library/Caches/Homebrew/downloads/19dbdc5c4ddcb06e2a0437e78cd551eebcb45db1bfdc8c8e0555a090d1396222--kubernetes-cli-1.32.0.bottle_manifest.json
==> Pouring kubernetes-cli--1.32.0.arm64_sequoia.bottle.tar.gz
🍺  /opt/homebrew/Cellar/kubernetes-cli/1.32.0: 255 files, 60.7MB
==> Installing ubuntu/microk8s/microk8s
==> python3 -m venv --system-site-packages --without-pip /opt/homebrew/Cellar/microk8s/2.3.4/libexec
==> python3 -m pip --python=/opt/homebrew/Cellar/microk8s/2.3.4/libexec/bin/python install urllib3==
==> python3 -m pip --python=/opt/homebrew/Cellar/microk8s/2.3.4/libexec/bin/python install /private/
==> Caveats
Run `microk8s install` to start with MicroK8s
==> Summary
🍺  /opt/homebrew/Cellar/microk8s/2.3.4: 876 files, 7.7MB, built in 29 seconds
==> Running `brew cleanup microk8s`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
==> Caveats
==> microk8s
Run `microk8s install` to start with MicroK8s
nari@narimac-mini ~ %

クラスタ作る箇所でmultipassがインストールされるんやけど、わけわかめなエラーが出る。

前にarm64のubuntu動かせたはずやのになぁ。

nari@narimac-mini ~ % microk8s install --cpu 4 --mem 6 --disk 50 --channel 1.30/stable --image lts ⭐️lts以外にoracularとかjammyをimageに指定してもアカン
Support for 'multipass' needs to be set up. Would you like to do that now? [y/N]: y
==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-cask/8df109b693b5b299b9fa69c57ba
############################################################################################# 100.0%
==> Downloading https://github.com/canonical/multipass/releases/download/v1.15.0/multipass-1.15.0+ma
==> Downloading from https://objects.githubusercontent.com/github-production-release-asset-2e65be/11
############################################################################################# 100.0%
==> Installing Cask multipass
==> Running installer for multipass with sudo; the password may be necessary. ⭐️前から思てたけど、他の仮想化でこんなん聞いてこんのに、microk8sのqemuはなんでsudoいるねん
Password:
installer: Package name is multipass
installer: Installing at base path /
installer: The install was successful.
🍺  multipass was successfully installed!
Waiting for multipass...
start failed: The following errors occurred:
Unexpected error in object_property_find_err() at ../../../qom/object.c:1330:
qemu-system-aarch64: Property 'host-arm-cpu.sme' not found ⭐️何言うてんねん、M4やったらアカンのかいな

launch failed: instance "microk8s-vm" already exists ⭐️何言うてんねん、今から作る指示してるのに、もうあるでってどういうことや
Warning: the "--mem" long option is deprecated in favour of "--memory". Please update any scripts, etc.
:(中略)
launch failed: instance "microk8s-vm" already exists 
An error occurred with the instance when trying to launch with 'multipass': returned exit code 2.
Ensure that 'multipass' is setup correctly and try again.
nari@narimac-mini ~ % 

multipassはbrewのcasks系で入るパッケージなのな。GUIで動くからアプリケーションの一覧に表示される。

その中で仮想マシンが管理されるようになってて、コマンドラインでmultipass deletemultipass purgeとかしてやり直したり、イメージ指定を変えてみたりしてもアカンかった。

rancher desktop(limaが裏で動いてる)とUTM(apple virtualizationが裏で動いてる)使ってるからかなぁ。

ぬぬぬ。

multipassロクなことない。途中で出てくるsudoも前から気に入らんかったから、multipass経由でmicrok8s使うのやめ。

UTMでapple virtualization使うarm64のubuntu作ったるねん。

UTMでubuntu行くで

arm64版のubuntuインストールメディアをダウンロードして用意する。デスクトップ版やなくてサーバ版な。

https://ubuntu.com/download/server/arm

UTMでホスト作る。

このへんでやったことと同じことするけど、ホスト名とIPを新規で割り当てておき宅内DNSにも登録しとく。

hostname:kubearm
IP: 192.168.1.116
subnet: 192.168.1.0/24
GW: 192.168.1.1
dns: 172.16.17.15,8.8.8.8

ホスト作るとこうなる。

arm64-microk8s-kubearm
arm64-microk8s-kubearm
arm64-microk8s-kubearm
arm64-microk8s-kubearm

下の段がx86のmicrok8sホストで、上の段が今回のarm64のホスト。

起動速度比べたら、なんやこれ。

arm64-microk8s-kubearm

セカンドやなくて、ミリセカンドって書いとるやないか。

コア数とメモリもちゃんと確保できとるんか確認。

┌nmon─16p──────[H for help]───Hostname=kubearm──────Refresh= 2secs ───05:47.12─888888888┐
│ CPU Utilisation ─                                                                     │
│---------------------------+-------------------------------------------------+         │
│CPU User%  Sys% Wait%  Idle|0          |25         |50          |75       100|         │
│  1   0.5   1.5   0.0  98.0|>                                                |         │
│  2   1.0   1.0   0.0  97.9|>                                                |         │
│  3   0.5   1.0   0.0  98.5|  >                                              |         │
│  4   1.0   0.5   0.0  98.5|>                                                |         │
│  5   1.5   1.0   0.0  97.4|>                                                |         │
│  6   1.0   1.0   0.0  98.0|>                                                |         │
│---------------------------+-------------------------------------------------+         │
│Avg   0.9   0.9   0.1  98.0|>                                                |         │
│---------------------------+-------------------------------------------------+         │
│ Memory and Swap ─                                                                     │
│ PageSize:4KB   RAM-Memory  Swap-Space       High-Memory     Low-Memory                │
│ Total (MB)         9920.2      4096.0       - not in use   - not in use               │
│ Free  (MB)         7503.9      4096.0                                                 │
│ Free Percent         75.6%      100.0%                                                │
│ Linux Kernel Internal Memory (MB)                                                     │
│                        Cached=    1363.8     Active=    1655.7                        │
│ Buffers=      31.7 Swapcached=       0.0  Inactive =     533.7                        │
│ Dirty  =       0.2 Writeback =       0.0  Mapped   =     509.3                        │
│ Slab   =     110.5 Commit_AS =    2872.2 PageTables=      10.1                        │
│ Disk I/O ──/proc/diskstats──Requested KB/s────Warning:may contains duplicates─ssssssss│
│DiskName Busy  Read Write  |0          |25         |50          |75       100|         │
│Totals Read-MB/s=0.0      Writes-MB/s=1.1      Transfers/sec=159.8                     │
│─99999999999999999999999999999999999999999999999999999999999999999999999999999─ssssssss│
│                                                                                       │
│                                                                                       │
└─aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa┘

なんとか動いたな。Pod動いてへん状態でも2.4GBほどメモリ使うんやな。

宅内DNSの参照

クラスタのPodに宅内DNSで名前引きさせたい。

前はDNSの設定をクラスタにつけるんやなくて、Podにホスト名定義つけとった。

そやけど、x86のdockerコンテナ・x86のkubernetes・macminiのnginxとrancherの中のdockerコンテナ・UTMの中で動くmacとkubernetesって扱うと5つほどhostnamesをマニフェストに書かなアカン。

microk8sのサイト見たら、参照させる方法あるらしい。

MicroK8s - Add-on dns | MicroK8s
MicroK8s is the simplest production-grade upstream K8s. Lightweight and focused. Single command install on Linux, Window...

なるほど、やってみた。

microk8s enable dns:172.16.17.15

設定できてるか確認するときこうする。

microk8s kubectl -n kube-system edit configmap/coredns

そしたらvisudoみたいな感じで編集できる画面になるんやけど、その中に宅内DNSの設定してある箇所が見える。

apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health {
          lameduck 5s
        }
        ready
        log . {
          class error
        }
        kubernetes cluster.local in-addr.arpa ip6.arpa {
          pods insecure
          fallthrough in-addr.arpa ip6.arpa
        }
        prometheus :9153
        forward .  172.16.17.15 ⭐️入っとる!!
        cache 30
        loop
        reload
        loadbalance
    }

たぶんdns設定は複数書けるんやろけど、宅内DNSから8.8.8.8とか解決できるようにしといた。

クラスタ作る

やってみた。ちゃんとできとる。

root@kubearm:/microk8s/log# cat kube.log 
---Recreate start---
-------- 2025-01-15_05:33:17 --------
microk8s removed
-------- 2025-01-15_05:33:42 --------
microk8s (1.32/stable) v1.32.0 from Canonical** installed
Enabling default storage class.
WARNING: Hostpath storage is not suitable for production environments.
         A hostpath volume can grow beyond the size limit set in the volume claim manifest.

deployment.apps/hostpath-provisioner created
storageclass.storage.k8s.io/microk8s-hostpath created
serviceaccount/microk8s-hostpath created
clusterrole.rbac.authorization.k8s.io/microk8s-hostpath created
clusterrolebinding.rbac.authorization.k8s.io/microk8s-hostpath created
Storage will be available soon.
namespace/container-registry created
persistentvolumeclaim/registry-claim created
deployment.apps/registry created
service/registry created
configmap/local-registry-hosting configured
The registry will be created with the size of 30Gi.
Default storage class will be used.
Addon core/hostpath-storage is already enabled
Setting 10.0.1.1 as host-access
Disabling DNS
Reconfiguring kubelet
Removing DNS manifest
deployment.apps "coredns" deleted
serviceaccount "coredns" deleted
configmap "coredns" deleted
service "kube-dns" deleted
clusterrole.rbac.authorization.k8s.io "coredns" deleted
clusterrolebinding.rbac.authorization.k8s.io "coredns" deleted
DNS is disabled
Enabling DNS
Will use  172.16.17.15 as upstream nameservers ⭐️宅内DNS参照の設定できてるっぽい
Applying manifest
serviceaccount/coredns created
configmap/coredns created
deployment.apps/coredns created
service/kube-dns created
clusterrole.rbac.authorization.k8s.io/coredns created
clusterrolebinding.rbac.authorization.k8s.io/coredns created
CoreDNS service deployed with IP address 10.152.183.10
Restarting kubelet
DNS is enabled
-------- kubernetes cluster created --------
-------- 2025-01-15_05:34:58 --------

kubernetesのノードできとるか確認。

root@kubearm:~# kubectl get nodes 
NAME      STATUS   ROLES    AGE   VERSION
kubearm   Ready    <none>   14m   v1.32.0
root@kubearm:~# 

Podのマニフェストは基本そのままで使うんやけど、https提供するリバプロにホスト名書いてるから、ここは書き換える。

arm64-microk8s-kubearm

以下、更新したマニフェストの箇所。

spec:
  containers:
    - env:
        - name: DOMAINS
          value: kubearm.intra.gavann-it.com -> http://sv-django:38080 ⭐️ここにあるsv-djangoへのホスト名をkubearmに更新した
        - name: STAGE
          value: local
        - name: CLIENT_MAX_BODY_SIZE
          value: 20M

dockerイメージをインポート

teratermマクロ書き換えて、rancher desktopで作ったdockerイメージの元ファイルをarm64のkubernetes環境へmicrok8s.ctr images importさせる。

nari@kubearm:~$ sudo su -
[sudo] password for nari:
root@kubearm:~# cd /data
root@kubearm:/data# rm -fR gvis-pv-ubun ; sync ; cp /microk8s/nariDockerDat/gvis-pv-ubun.tar.gz /data ; tar xzf gvis-pv-ubun.tar.gz
root@kubearm:/data# chown -R nari:nari gvis-pv-ubun ; chmod -R 777 gvis-pv-ubun ; sync
root@kubearm:/data# cd /microk8s/nariDockerDat/DockerImages
root@kubearm:/microk8s/nariDockerDat/DockerImages# microk8s.ctr images rm docker.io/library/save-django:gvis-saved
docker.io/library/save-django:gvis-saved
root@kubearm:/microk8s/nariDockerDat/DockerImages# microk8s.ctr images rm docker.io/library/save-xrdpubu:gvis-saved
WARN[0000] docker.io/library/save-xrdpubu:gvis-saved: image not found
root@kubearm:/microk8s/nariDockerDat/DockerImages# microk8s.ctr images import save-django.tar ⭐️1分ぐらいかかる
unpacking docker.io/library/save-django:gvis-saved (sha256:359d319dba03fcc641e57a7768ad5953b59577765511f476c02518fd555c75f2)...done
root@kubearm:/microk8s/nariDockerDat/DockerImages# microk8s.ctr images import save-xrdpubu.tar ⭐️3分ぐらいかかる
unpacking docker.io/library/save-xrdpubu:gvis-saved (sha256:b71db51f90b2d1c978efd757572371df19bffd840795a99b5af1ecb7f707bcd9)...done
root@kubearm:/microk8s/nariDockerDat/DockerImages# microk8s.ctr images ls | grep save ⭐️インポートしたdockerイメージ以外にも、ctrのイメージ置き場にはkubernetesのコンポーネントいっぱいあるからgrepさせて表示
docker.io/library/save-django:gvis-saved                                                                              application/vnd.oci.image.manifest.v1+json                sha256:359d319dba03fcc641e57a7768ad5953b59577765511f476c02518fd555c75f2 1.1 GiB   linux/arm64                                                                                           io.cri-containerd.image=managed
docker.io/library/save-xrdpubu:gvis-saved                                                                             application/vnd.oci.image.manifest.v1+json                sha256:b71db51f90b2d1c978efd757572371df19bffd840795a99b5af1ecb7f707bcd9 5.4 GiB   linux/arm64                                                                                           io.cri-containerd.image=managed
root@kubearm:/microk8s/nariDockerDat/DockerImages#

ctrでイメージのリスト出力させたら横に長いなぁ。

永続化領域をコピー

mariadbのデータとdjangoな。

mariadbのデータ流し込み

mariadbとdjangoのデータ流し込んでく。teratermマクロ使ってscpで流し込みからpodの再作成まで。

nari@kubearm:~$ sudo su -
[sudo] password for nari:
root@kubearm:~# rm -f /tmp/*.sql ; rm -f /tmp/gvis.cnf
root@kubearm:~# ps -ef |grep -v grep |grep -c scp
2
root@kubearm:~# ps -ef |grep -v grep |grep -c scp
0
root@kubearm:~# echo SCP finish
SCP finish
root@kubearm:~# m8
root@kubearm:/microk8s# kubectl delete -f sv-mariadb-pod.yaml
kubectl delete -f gvis-PersistentVol-mariadbconf.yaml
Error from server (NotFound): error when deleting "sv-mariadb-pod.yaml": pods "sv-mariadb" not found
root@kubearm:/microk8s# kubectl delete -f gvis-PersistentVol-mariadbconf.yaml
kubectl delete -f gvis-PersistentVol-mariadb.yaml
persistentvolume "gvis-pv-mariadbconf" deleted
persistentvolumeclaim "gvis-pv-mariadbconf-claim" deleted
root@kubearm:/microk8s# kubectl delete -f gvis-PersistentVol-mariadb.yaml
persistentvolume "gvis-pv-mariadb" deleted
persistentvolumeclaim "gvis-pv-mariadb-claim" deleted
root@kubearm:/microk8s# cd /data
root@kubearm:/data# rm -fR ./gvis-pv-mariadb ; rm -fR ./gvis-pv-mariadbconf ; sync ; sync
root@kubearm:/data# mkdir -p /data/gvis-pv-mariadbconf/nari/fullback/ ; mkdir -p gvis-pv-mariadb
root@kubearm:/data# cd /microk8s/nariDockerDat/
root@kubearm:/microk8s/nariDockerDat# cp -p  /tmp/gvis.cnf                                             /microk8s/nariDockerDat/sv_mariadb11conf/
root@kubearm:/microk8s/nariDockerDat# mv     /tmp/gvis.cnf                                             /data/gvis-pv-mariadbconf/
root@kubearm:/microk8s/nariDockerDat# cp -p ./sv_mariadb11conf/nari/fullback/2_fullRecover.sh          /data/gvis-pv-mariadbconf/nari/fullback/
root@kubearm:/microk8s/nariDockerDat# cp -p ./sv_mariadb11conf/nari/fullback/4_nariDB_DjangoRecover.sh /data/gvis-pv-mariadbconf/nari/fullback/
root@kubearm:/microk8s/nariDockerDat# mv    /tmp/FullBackup_nariDB_1st.sql                             /data/gvis-pv-mariadbconf/nari/
root@kubearm:/microk8s/nariDockerDat# mv    /tmp/FullBackup_nariDB_Django.sql                          /data/gvis-pv-mariadbconf/nari/
root@kubearm:/microk8s/nariDockerDat# cd /data
root@kubearm:/data# chmod -R 777 gvis-pv-mariadbconf
root@kubearm:/data# chmod -R 777 gvis-pv-mariadb
root@kubearm:/data# chmod 644 /data/gvis-pv-mariadbconf/gvis.cnf
root@kubearm:/data# m8
root@kubearm:/microk8s# kubectl create -f gvis-PersistentVol-mariadbconf.yaml
persistentvolume/gvis-pv-mariadbconf created
persistentvolumeclaim/gvis-pv-mariadbconf-claim created
root@kubearm:/microk8s# kubectl create -f gvis-PersistentVol-mariadb.yaml
persistentvolume/gvis-pv-mariadb created
persistentvolumeclaim/gvis-pv-mariadb-claim created
root@kubearm:/microk8s# kubectl create -f sv-mariadb-pod.yaml
pod/sv-mariadb created
root@kubearm:/microk8s# sync ; sync ; sleep 30 ⭐️mariadbの初期化ってだいたい30秒ぐらいで終わるから、少し待たせる

podにログインさせてデータベースのロードする。合間にsleep入れといて起動の間合いをとる。

root@kubearm:/microk8s# kubectl exec -it `kubectl get pod | grep mariadb | awk '{print $1}'` -- bash
root@svmariadb:/# sync ; sleep 60 ; sync
root@svmariadb:/# /bin/sh /etc/mysql/conf.d/nari/fullback/2_fullRecover.sh
root@svmariadb:/# sync ; sleep 20 ; sync
root@svmariadb:/# /bin/sh /etc/mysql/conf.d/nari/fullback/4_nariDB_DjangoRecover.sh ⭐️テストデータのは使わんけど、まぁ入れとく
root@svmariadb:/# sync ; sleep 20 ; sync
root@svmariadb:/# mariadb -unari -pboo
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 5
Server version: 11.4.4-MariaDB-ubu2404-log mariadb.org binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> show variables like 'max_allowed_packet' ;
+--------------------+------------+
| Variable_name      | Value      |
+--------------------+------------+
| max_allowed_packet | 1073741824 | ⭐️mariadbの設定ファイル(/etc/mysql/conf.dへマウント)が反映されとる
+--------------------+------------+
1 row in set (0.001 sec)

MariaDB [(none)]> show databases ;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| nariDB_1st         | ⭐️データベースリストアできとる
| nariDB_Django      |
+--------------------+
3 rows in set (0.000 sec)

MariaDB [(none)]> use nariDB_1st ;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MariaDB [nariDB_1st]> select count(*) from GVIS_keihi ;
+----------+
| count(*) |
+----------+
|    11790 | ⭐️主たるテーブルにデータ入っとる
+----------+
1 row in set (0.004 sec)

MariaDB [nariDB_1st]> exit
Bye
root@svmariadb:/# exit
exit
root@kubearm:/microk8s# rm -f /data/gvis-pv-mariadbconf/nari/FullBackup_nariDB_1st.sql
root@kubearm:/microk8s# rm -f /data/gvis-pv-mariadbconf/nari/FullBackup_nariDB_Django.sql
root@kubearm:/microk8s#

djangoのデータ流し込み

nari@kubearm:~$ sudo su -
[sudo] password for nari:
root@kubearm:~# rm -f /tmp/sv_django-uwsgi-nginx.tar.gz
root@kubearm:~# ps -ef |grep -v grep |grep -c scp
0
root@kubearm:~# echo SCP finish
SCP finish
root@kubearm:~# cd /microk8s/nariDockerDat ; rm -f sv_django-uwsgi-nginx.tar.gz ; mv /tmp/sv_django-uwsgi-nginx.tar.gz ./
rm -fR ./sv_django-uwsgi-nginx
root@kubearm:/microk8s/nariDockerDat# rm -fR ./sv_django-uwsgi-nginx
root@kubearm:/microk8s/nariDockerDat# m8
root@kubearm:/microk8s# kubectl delete -f sv-django-pod.yaml
kubectl delete -f gvis-PersistentVol-sv_django-uwsgi-nginx.yaml
root@kubearm:/microk8s# kubectl delete -f gvis-PersistentVol-sv_django-uwsgi-nginx.yaml
persistentvolume "gvis-pv-django-uwsgi-nginx" deleted
persistentvolumeclaim "gvis-pv-django-uwsgi-nginx-claim" deleted
root@kubearm:/microk8s# cd /data
root@kubearm:/data# rm -fR ./gvis-pv-django-uwsgi-nginx ; rm -fR ./gvis-pv-django-sslcerts ; sync
root@kubearm:/data# cp -p /microk8s/nariDockerDat/sv_django-uwsgi-nginx.tar.gz ./
root@kubearm:/data# tar xzf sv_django-uwsgi-nginx.tar.gz
root@kubearm:/data# mv ./sv_django-uwsgi-nginx/app ./gvis-pv-django-uwsgi-nginx ; mkdir gvis-pv-django-sslcerts
root@kubearm:/data# /bin/sh /data/sv_django-uwsgi-nginx/kubearmCopy.sh
root@kubearm:/data# chmod 777 gvis-pv-django-uwsgi-nginx ; chmod 777 gvis-pv-django-sslcerts
root@kubearm:/data# rm -fR ./sv_django-uwsgi-nginx/
root@kubearm:/data# m8
root@kubearm:/microk8s# kubectl create -f gvis-PersistentVol-sv_django-uwsgi-nginx.yaml
persistentvolume/gvis-pv-django-uwsgi-nginx created
persistentvolumeclaim/gvis-pv-django-uwsgi-nginx-claim created
root@kubearm:/microk8s# kubectl create -f sv-django-pod.yaml
pod/sv-django created
root@kubearm:/microk8s# sleep 10 ⭐️django起動まで少しだけ待たせる
root@kubearm:/microk8s# kubectl exec -it `kubectl get pod | grep sv-django | awk '{print $1}'` -- bash
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
root@sv-django:/# pip3 list -o ⭐️更新溜まってきたらrancher desktopにある原本のpython環境でpip3するようにしてる
Package    Version Latest Type
---------- ------- ------ -----
kiwisolver 1.4.7   1.4.8  wheel
matplotlib 3.9.4   3.10.0 wheel
numpy      2.2.0   2.2.1  wheel
pillow     11.0.0  11.1.0 wheel
pyparsing  3.2.0   3.2.1  wheel
setuptools 75.6.0  75.8.0 wheel
root@sv-django:/# exit
exit
root@kubearm:/microk8s# sc
root@kubearm:/microk8s/script# sh ./415_ReCreateHTTPSpod.sh ⭐️TLSリバプロのpod作り直し
NAME         READY   STATUS    RESTARTS   AGE
sv-django    1/1     Running   0          15s
sv-mariadb   1/1     Running   0          10m
pod/sv-https-portal created
NAME              READY   STATUS              RESTARTS   AGE
sv-django         1/1     Running             0          15s
sv-https-portal   0/1     ContainerCreating   0          0s
sv-mariadb        1/1     Running             0          10m
root@kubearm:/microk8s/script#

ステータスはcreatingとか出てるけど、10秒ぐらいでpodは起動して使える状態になる。

xrdpのpod起動する

これもすぐ動く。

root@kubearm:/microk8s/script# cat 413_ReCreateXRDPpod.sh 
#!/bin/sh
## -------------------------------------------------------------------------
## Script Name  : 413_ReCreateXRDPpod.sh
##  Created by  : T.Naritomi
##          on  : 2024.12.24
##  Updated by  : 
##          on  :
##  Parameters  : 
##  Return Code : 0=Normal End
##     Comments : minikube -> microk8s
## -------------------------------------------------------------------------

KB_HOME=/microk8s

## ---detail---------------------------------------------------------------- 

microk8s kubectl get pod

microk8s kubectl delete -f ${KB_HOME}/cl-ubun-pod.yaml
microk8s kubectl apply -f ${KB_HOME}/cl-ubun-pod.yaml

microk8s kubectl get pod

exit
root@kubearm:/microk8s/script# sh ./413_ReCreateXRDPpod.sh 
NAME              READY   STATUS    RESTARTS   AGE
sv-django         1/1     Running   0          7m31s
sv-https-portal   1/1     Running   0          7m16s
sv-mariadb        1/1     Running   0          17m
pod/cl-ubun created
NAME              READY   STATUS              RESTARTS   AGE
cl-ubun           0/1     ContainerCreating   0          1s
sv-django         1/1     Running             0          7m32s
sv-https-portal   1/1     Running             0          7m17s
sv-mariadb        1/1     Running             0          17m
root@kubearm:/microk8s/script#

フォワーディング

これやっとかなkubernetes稼働ホスト以外のホストから接続できん。

root@kubearm:/microk8s/script# cat 306_allForwardingStart.sh 
## -------------------------------------------------------------------------
## Script Name  : 306_allForwardingStart.sh
##  Created by  : T.Naritomi
##          on  : 2024.12.24
##  Updated by  :
##          on  :
##  Parameters  :
##  Return Code : 0=Normal End
##     Comments :
## -------------------------------------------------------------------------
## ---detail----------------------------------------------------------------

microk8s kubectl port-forward --address 0.0.0.0 `microk8s kubectl get pod | grep mariadb   | awk '{print $1}'` 13306:3306 &

microk8s kubectl port-forward --address 0.0.0.0 `microk8s kubectl get pod | grep cl-ubun   | awk '{print $1}'` 33389:3389 &

microk8s kubectl port-forward --address 0.0.0.0 `microk8s kubectl get pod | grep sv-django | awk '{print $1}'` 38080:8080 &

microk8s kubectl port-forward --address 0.0.0.0 `microk8s kubectl get pod | grep sv-https  | awk '{print $1}'` 30443:443 &

exit $?
root@kubearm:/microk8s/script# sh ./306_allForwardingStart.sh 
root@kubearm:/microk8s/script# Forwarding from 0.0.0.0:33389 -> 3389
Forwarding from 0.0.0.0:38080 -> 8080
Forwarding from 0.0.0.0:30443 -> 443
Forwarding from 0.0.0.0:13306 -> 3306

root@kubearm:/microk8s/script# 

Podの再作成したら、フォワーディングもやり直さなあかんで。

宅内DNS参照できとるんか

smbのファイルサーバ参照させたり、mariadbクライアント入れてて、kubernetesの閉じた環境内で動作確認するのに使うのがxrdpのPod。

nslookupも使えるようにしてる。

Podにログインして名前解決の確認してみる。

root@kubearm:/microk8s/script# cat 403_LoginXRDPpod.sh
#!/bin/sh
## -------------------------------------------------------------------------
## Script Name  : 403_LoginXRDPpod.sh
##  Created by  : T.Naritomi
##          on  : 2024.12.24
##  Updated by  :
##          on  :
##  Parameters  :
##  Return Code : 0=Normal End
##     Comments :
## -------------------------------------------------------------------------

## ---detail----------------------------------------------------------------

microk8s kubectl exec -it `microk8s kubectl get pod | grep cl-ubun | awk '{print $1}'` -- bash

exit
root@kubearm:/microk8s/script# sh ./403_LoginXRDPpod.sh ⭐️xrdpのPodにコマンドラインでログインする
root@clubu:/#
root@clubu:/# uname -n
clubu
root@clubu:/# cat /etc/hosts ⭐️マニフェストから削除したホスト名定義が消えてることを確認
# Kubernetes-managed hosts file.
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
fe00::0 ip6-mcastprefix
fe00::1 ip6-allnodes
fe00::2 ip6-allrouters
10.1.103.149    clubu
root@clubu:/#
root@clubu:/# nslookup narimac-mini.intra.gavann-it.com | tail -4
Name:   narimac-mini.intra.gavann-it.com
Address: 192.168.1.35 ⭐️名前解決できとる
** server can't find narimac-mini.intra.gavann-it.com: NXDOMAIN

root@clubu:/# nslookup kubearm.intra.gavann-it.com | tail -4
Name:   kubearm.intra.gavann-it.com
Address: 192.168.1.116 ⭐️名前解決できとる
** server can't find kubearm.intra.gavann-it.com: NXDOMAIN

root@clubu:/# nslookup gvis-mac.intra.gavann-it.com | tail -4
Name:   gvis-mac.intra.gavann-it.com
Address: 192.168.1.117 ⭐️名前解決できとる
** server can't find gvis-mac.intra.gavann-it.com: NXDOMAIN

root@clubu:/# nslookup nafslinux.intra.gavann-it.com | tail -4
Name:   nafslinux.intra.gavann-it.com
Address: 192.168.1.118 ⭐️名前解決できとる
** server can't find nafslinux.intra.gavann-it.com: NXDOMAIN

root@clubu:/# nslookup kubelinux.intra.gavann-it.com | tail -4
Name:   kubelinux.intra.gavann-it.com
Address: 192.168.1.119 ⭐️名前解決できとる
** server can't find kubelinux.intra.gavann-it.com: NXDOMAIN

root@clubu:/#

前からそうなんやけど、このフォワーディングってのがときどきエラー吐く。

root@kubearm:/microk8s/script# Handling connection for 33389
E0115 07:44:58.599585  163508 portforward.go:424] "Unhandled Error" err="an error occurred forwarding 33389 -> 3389: error forwarding port 3389 to pod 2964851cac7aed6612dcfd9ffdeb7dac681a93679c3554eda530aabe61b23e52, uid : failed to find sandbox \"2964851cac7aed6612dcfd9ffdeb7dac681a93679c3554eda530aabe61b23e52\" in store: not found"
error: lost connection to pod

root@kubearm:/microk8s/script#

エラー出ると、フォワーディングいったん停止してもう1回フォワーディングせな繋がらんのよな。

root@kubearm:/microk8s/script# sh ./305_allForwardingStop.sh 
root@kubearm:/microk8s/script# sh ./306_allForwardingStart.sh 
root@kubearm:/microk8s/script# Forwarding from 0.0.0.0:38080 -> 8080
Forwarding from 0.0.0.0:33389 -> 3389
Forwarding from 0.0.0.0:30443 -> 443
Forwarding from 0.0.0.0:13306 -> 3306

root@kubearm:/microk8s/script# cat 305_allForwardingStop.sh 
## -------------------------------------------------------------------------
## Script Name  : 305_allForwardingStop.sh
##  Created by  : T.Naritomi
##          on  : 2024.12.24
##  Updated by  : 
##          on  :
##  Parameters  :
##  Return Code : 0=Normal End
##     Comments : change driver hyperkit -> qemu2 , minikube -> microk8s
## -------------------------------------------------------------------------
## ---detail----------------------------------------------------------------

## stop port forward
ps -ef | grep kubectl | grep port-forward | grep -v dashboard | awk {'print $2'} | xargs kill -9

exit $?
root@kubearm:/microk8s/script# cat 306_allForwardingStart.sh 
## -------------------------------------------------------------------------
## Script Name  : 306_allForwardingStart.sh
##  Created by  : T.Naritomi
##          on  : 2024.12.24
##  Updated by  :
##          on  :
##  Parameters  :
##  Return Code : 0=Normal End
##     Comments :
## -------------------------------------------------------------------------
## ---detail----------------------------------------------------------------

microk8s kubectl port-forward --address 0.0.0.0 `microk8s kubectl get pod | grep mariadb   | awk '{print $1}'` 13306:3306 &

microk8s kubectl port-forward --address 0.0.0.0 `microk8s kubectl get pod | grep cl-ubun   | awk '{print $1}'` 33389:3389 &

microk8s kubectl port-forward --address 0.0.0.0 `microk8s kubectl get pod | grep sv-django | awk '{print $1}'` 38080:8080 &

microk8s kubectl port-forward --address 0.0.0.0 `microk8s kubectl get pod | grep sv-https  | awk '{print $1}'` 30443:443 &

exit $?
root@kubearm:/microk8s/script# 

ようやっとx86/armのdocker/kubernetesの環境維持できるようになった。

タイトルとURLをコピーしました