time="2025-04-16T17:46:45+09:00" level=debug msg="retry loop: attempt 9"
time="2025-04-16T17:46:45+09:00" level=debug msg="Running SSH command: <hidden>"
time="2025-04-16T17:46:45+09:00" level=debug msg="SSH command succeeded"
time="2025-04-16T17:46:45+09:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2025-04-16T17:46:47+09:00" level=debug msg="retry loop: attempt 10"
time="2025-04-16T17:46:47+09:00" level=debug msg="Running SSH command: <hidden>"
time="2025-04-16T17:46:47+09:00" level=debug msg="SSH command succeeded"
time="2025-04-16T17:46:47+09:00" level=debug msg="error: Temporary error: pull secret not updated to disk - sleeping 2s"
time="2025-04-16T17:46:49+09:00" level=debug msg="retry loop: attempt 11"
time="2025-04-16T17:46:49+09:00" level=debug msg="Running SSH command: <hidden>"
time="2025-04-16T17:47:15+09:00" level=debug msg="Failed to create new ssh session: ssh: unexpected packet in response to channel open: <nil>"
time="2025-04-16T17:47:15+09:00" level=debug msg="SSH command failed"
time="2025-04-16T17:47:15+09:00" level=debug msg="non-retriable error: failed to read /var/lib/kubelet/config.json file: ssh command error:\ncommand : sudo cat /var/lib/kubelet/config.json\nerr : ssh: unexpected packet in response to channel open: <nil>\n: "
time="2025-04-16T17:47:15+09:00" level=debug msg="Running 'sw_vers -productVersion'"
ssh: unexpected packet in response to channel open
ps aux | grep vmnetd # Verified Docker network service
sudo launchctl bootout system /...vmnetd.plist # Disabled it for current session
sudo rm -f /Library/LaunchDaemons/...vmnetd.plist # Prevented it from auto-starting
crc start # CRC now boots and works properly!
ps aux | grep vmnetd # Verified Docker network service
sudo launchctl bootout system /...vmnetd.plist # Disabled it for current session
sudo rm -f /Library/LaunchDaemons/...vmnetd.plist # Prevented it from auto-starting
crc start # CRC now boots and works properly!
Let's approach this clearly, methodically, and comprehensively, as you've hit a very stubborn and persistent issue.
Your Current Issue (Recap):
The error you're seeing:
lua
Copy
Edit
Temporary error: pull secret not updated to disk
...
failed to read /var/lib/kubelet/config.json file: ssh command error:
err : Process exited with status 1
: /bin/bash: Input/output error
Means clearly:
CRC VM starts and runs
SSH succeeds initially
CRC repeatedly tries and fails to read or write the pull secret on disk
Eventually fails with a low-level /bin/bash: Input/output error
Potential Root Causes (What we've learned so far):
Possible Cause Likelihood (Your case) Explanation
Corrupted CRC Disk Image Very Likely Frequent cause after repeated CRC restarts or host interruption
Docker Desktop interference Very Likely vmnetd can corrupt vfkit networking (confirmed earlier)
Corporate VPN/Firewall Likely Blocks 127.0.0.1 or internal traffic silently
Insufficient RAM / CPU Possible Less likely, but can cause silent failures
macOS permissions Possible Disk permissions or macOS system policy issues
Let's Do Comprehensive, Step-by-Step Troubleshooting:
Run these exact steps to pinpoint the issue.
STEP 1: Fully reset CRC again (with extra steps)
bash
Copy
Edit
crc stop
crc delete --force
crc cleanup
rm -rf ~/.crc
rm -rf ~/Library/Application\ Support/crc
rm -rf ~/.kube
rm -rf ~/.config/crc
STEP 2: Check and completely disable Docker Desktop Networking
bash
Copy
Edit
ps aux | grep vmnetd
If you see Docker again, remove it completely for now:
bash
Copy
Edit
sudo launchctl bootout system /Library/LaunchDaemons/com.docker.vmnetd.plist
sudo rm -f /Library/LaunchDaemons/com.docker.vmnetd.plist
Also quit Docker Desktop fully.
STEP 3: Disconnect VPN or Corporate Network (Critical Test)
Switch temporarily to your personal Wi-Fi hotspot (phone), just for this test.
Confirm no corporate firewall/VPN is active.
STEP 4: Reboot Your Mac (Absolutely critical step)
This ensures a fully clean vfkit/hypervisor environment.
STEP 5: Re-run CRC Setup Completely Fresh
bash
Copy
Edit
crc setup
crc start --memory 12288 --cpus 4 --pull-secret ~/Downloads/pull-secret.txt
(12GB RAM, 4 CPUs for stable startup)
STEP 6: If CRC still fails again, immediately do this:
Check the internal status inside CRC VM:
bash
Copy
Edit
crc ssh
Inside VM:
Check kubelet service:
bash
Copy
Edit
sudo systemctl status kubelet
sudo journalctl -xe -u kubelet
Check kubelet directory and file presence:
bash
Copy
Edit
ls -la /var/lib/kubelet/config.json
sudo cat /var/lib/kubelet/config.json
Check disk issues inside VM (Critical):
bash
Copy
Edit
dmesg | grep -i error
sudo journalctl -xe | grep -i error
STEP 7: Network Checks inside CRC VM (critical test):
bash
Copy
Edit
ping -c4 google.com
curl https://mirror.openshift.com
cat /etc/resolv.conf
ip route
Interpreting Results (most likely outcomes):
Test If passes If fails
Ping Google (VM network test) Networking fine
VPN/Firewall blocks VM access
curl mirror.openshift.com External DNS fine
DNS broken or network blocked by firewall
kubelet status CRC VM internal services OK
VM internal service failed due to config error
ls /var/lib/kubelet/config.json File present
File missing or corrupted, CRC bootstrap failed
dmesg grep error Clean/no disk errors
Final Diagnosis (Your next action depends on these results):
If disk errors or file missing: CRC VM image corrupted โ full reinstall (CRC + reboot)
If ping/curl fail: VPN/firewall โ retry on personal hotspot
If kubelet/journalctl show errors: CRC internal issue (likely image or CRC bug)
Recommended Final Solutions (pick based on result):
Fully remove CRC and Docker Desktop, reboot, reinstall CRC completely fresh.
Run CRC only when disconnected from any corporate VPN or firewall.
Try CRC with slightly older stable bundle (like 4.13.x if 4.18.x fails repeatedly on your system).